aaa Lame 


Matrix Algebra 


FOR SOCIAL SCIENTISTS 


Matríx Algebra for Socíal Scíentísts 


Matríx Algebra 
for Socíal Scíentísts 


PAUL HORST 


University of Washington 


HOLT, RINEHART AND WINSTON, INC. 


New York Chicago San Francisco Toronto London 


Copyright (€) 1963 by Holt, Rinehart and Winston, Inc. 
АП rights reserved 

Library of Congress Catalog Card Number: 63-7368 
23733-0113 

Printed in the United States of America 


R Т, West Benga; 


1623-64 _. Ke 
№. RAG... 
519 . 296 


LAC 
HO! 


To Muriel 


Preface 


The purpose of this book is, essentially, to introduce students, teachers, 
and research workers without mathematical training to a branch of mathe- 
maties which is becoming increasingly valuable for the analysis of scientific 
data. It was written primarily for behavioral scientists with no more 
than a reasonably good working knowledge of high school freshman algebra 
who have passed an introductory course in statistics. The overriding 
concern in the selection, organization, and treatment of topics throughout 
the book is with the collection and processing of experimental data for the 
Purpose of answering questions crucial to the prediction and control 
of human behavior. 

The book is an outgrowth of two different kinds of experience. First, 
for ten years during and prior to World War II, while engaged in psycho- 
logical and personnel research work, I developed and adapted the more 
elementary concepts of matrix algebra for use by computing staffs con- 
sisting in large part of persons with no more than a high school education. 
Second, for seventeen years during and since the war I have taught the 
Principles of simple matrix algebra, together with its applications to the 
analysis of social science data, to students with only high school mathe- 
matics. Much of my treatment is not orthodox from the mathematician’s 
Point of view. In general, I have been willing to sacrifice mathematical 
elegance and rigor for the sake of simplicity and ease of comprehension, 
although I have not intentionally sacrificed mathematical accuracy. No 
attempt is made to cover the more advanced concepts of matrix theory. 
The following are what I regard as among the unique characteristics 
of the book: 


l. It is specifically oriented toward the three major types of analyses 
"Sed in the social sciences: (1) multiple regression analysis, (2) factor 
analysis, and (3) analysis of variance. 

2. It emphasizes the distinction, so important in the social Sciences 

tween data matrices and derived matrices. i 
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3. It emphasizes the distinction between entities and attributes in the 
matrix presentation of data. This is frequently a source of confusion to 
both students and mature scholars in the social sciences. 

4. In addition to the conventional notion of matrix multiplic. 
develops the concept of matrix multiplication as the sum of major 
of vectors. This concept is of great practical value in the 
social science data. 


ation, it 
produets 
analysis of 


5. The concept of rank from a computational point of view and entirely 
without the aid of determinants is developed. In fact, those elements 
of matrix algebra essential for the social sciences are presented entirely 
without the aid of determinants. 

6. The concept of nonsingular matrices is minimized, and 
primary emphasis is given to the general class of rectangula 
(basic matrices) whose rank is equal to the smaller dimension of t 

7. The concept of “basic structure" 
developed as the triple product from left 
nonhorizontal matrix orthonormal by col 
and (3) a nonvertical matrix orthonormal 
narrower related conce 
is subordinated. 

8. The concept of the 
that matrix which satisfi 
multiplied by its general 
from the identity matri 


, instead, 
r matrices 
he matrix. 
of any real vertical matrix is 
to right, respectively, of (1) a 
umns, (2) a diagonal matrix, 
by rows. The traditional and 
pt of latent roots and vectors of a square matrix 


general inverse of any real matrix is defined as 
es the following conditions: if any real matrix is 
inverse on either side and the product subtracted 


x, then the trace of the square of the difference 
matrix isa minimum. The conventional inverse of a nonsingular matrix 


is considered as a special case of the general inverse of a nonbasie rectan- 


gular matrix, and the importance of this concept for the analysis of data 
in the social sciences is emphasized. 


Chapter 1 develops the 
must be based on matrices of observed data. 
Chapter 2 shows how we indicate in symbolic form that tables of ob- 


Served data are to be considered as matrices to which the powerful methods 
of matrix algebra may be applied. 


Chapter 3 describes the 


point of view that all scientific investigations 


different kinds of matrices th. 


at are of special 
analysis of data, 


oncept of rank, which is 
of parsimony.” 
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Chapters 17 and 18 develop the concept of basic structure, which, 
because of its simplicity and greater usefulness in the analysis of social 
science data, is substituted for the more conventional mathematical con- 
cepts of latent roots and vectors. 

Chapters 19, 20, and 21 develop, in easy stages and for various special 
cases, the notion of matrix inversion, which is the basis for solving for 
the unknowns in scientific studies. 

Chapter 22 develops and presents applications of the general rank reduc- 
tion theorem, which is of great importance in the social sciences but which 
has received little emphasis in conventional texts on matrix algebra. 

Chapter 23 expands the conventional mathematical approach to the 
solution of linear equations to fit the real life problems which scientists 
are called upon to solve. 


The professional mathematician may become impatient with the lei- 
Surely pace of Part I and most of Part II. Also, he may experience 
irritation with the unconventional format and the unorthodox develop- 
ment for many of the proofs. If so, I beg his indulgence, for I can assure 
him that the traditional approach of the mathematician has negatively 
conditioned many students to the study of mathematics. It is this large 
group that I have sought to reassure and to persuade that mathematics 
is not necessarily frustrating, but can actually be interesting, entertaining, 
and above all, useful. Reactions to earlier dittoed and mimeographed 
Versions of the text seem to indicate that I have been at least partially 
successful. 

In particular, some mathematicians may become restive with the ex- 
tended treatments involving matrix transposition and supermatrices. If 
So, I invite them to become involved for a time in projects in which these 
Concepts are crucial for efficient data analysis operations. It is here that 
We see how concepts, simple in the abstract, can become surprisingly 
confusing in application. 

I have deliberately avoided the conventional practice of using boldface 
characters to distinguish matrices from scalar quantities. This reflects my 
Position that the matrix should be the primary operating unit rather than 
the scalar or vector, which are merely special cases of matrices. I prefer to 
define scalar quantities verbally whenever any ambiguity arises. In 
&eneral, I have been much more concerned than the professional mathe- 
Matician with the development of a functional, as distinguished from a 
descriptive, symbolic system. 

I owe a particular debt of gratitude to Professor Robert M. Thrall, 
Who patiently read the manuscript and made many valuable Suggestions. 
Most of these I was able to utilize to good advantage. There are, however 
Several points of disagreement between us. A brief discussion of {Rese 
may throw further light on the essential differences between my approach 
and that of the professional mathematicians. 
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The use of subscript notation to indicate the dimensions of a matrix is 
objectionable to Professor Thrall, whereas I have found it extremely useful 
in the manipulation of matrix expressions. It is true that ambiguities can 
arise, but I find it much better to clarify these for particular situations 
than to complicate the notation with additional symbols which, among 
other things, may cause confusion with computer programming notation. 
Also, in the case of the supermatrix notation in Chapter 11, Professor 
Thrall points out certain limitations. I confess that I am not completely 
satisfied with the notation as I have developed it thus far, 
in agreement, however, with Bodewi 
“make the notation do the work.” 
of supermatrices seems complex, 
supermatrices may be com 

themselves are simple. 

I am particularly indebted to 
to the work of E. H. 
This concept arises na 


I am thoroughly 
g that a major objective should be to 
If the notation for the multiplieation 
, it is because data analysis involving 
plex and confusing, even though the concepts 


Professor Thrall for calling my attention 
Moore and his students on the "general inverse." 
turally from the definition of the basic structure of 
а matrix. Since my own independent invention of the concept in 1941 
I have found it very useful in the analysis of behavioral science data and 


I have been repeatedly surprised that mathematicians have not generally 
been familiar with it. 


Another significant comme: 
proliferation resulting from t 
matrices. This comment i 
the great gulf between pure 
the early work on matrix theory, 
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checked the exercises and answers. I, however, assume responsibility for 
such errors as may remain. 

Finally, I wish to express my deep appreciation for the financial support 
provided by the Office of Naval Research Contract Nonr-477(08) and 
Public Health Research Grant M-743(C6). 

P.H. 
Seattle, Washington 
1962 
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Simple Matrix Concepts 


Chapter 1 


Introductíon 


1.4 What a Matrix Is 


It is often true that simple dictionary definitions don't help you much to 
understand the meaning of words. This is certainly the case for the word 
"matrix." Anabridged Webster's dictionary defines matrix as “a cavity in 
which anything is formed such as a mold or die.” This definition has little 
to do with the way we shall use the word, even though it is adequate for one 
of the most common uses. The mathematicians have used matrix to mean 
something quite different. However, since we are going to discuss pretty 
much the same things as the mathematicians do when they speak of a 
“matrix,” we shall use the same word. 

The word matrix comes from the Latin, like a good many technical or 
scholarly words. The plural is therefore the Latin plural, matrices. Thus a 
quite simple concept is called by a rather technical name. 

Basically, a matrix, as we shall use the word, is a table of numbers with 
80 many rows and so many columns. Figure 1.1.1 is an example of a 
matrix. 


4; 1 
62 5 
13 4 
79 5 
Fie. 1.1.1 


You will notice that Fig. 1.1.1 has four rows and three columns. 
Another example of a matrix is Fig. 1.1.2. 


5 9 12 14 
6 8 13 15 
14 3 13 10 
15 16 8 13 
14 9 3 1l 
Fra. 1.1.2 
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You see that Fig. 1.1.2 has five rows and four columns. 

You can have as many or as few rows and columns in а matrix as you 
wish, although usually there are special reasons for deciding how many 
rows and how many columns you will have. You could, for example, have 
a matrix with only one row and one column, in which case you would have, 
of course, only a single number. 

The numbers used in a matrix are real numbers, numbers in the ordinary 
sense. Some of you doubtless know about imaginary and complex numbers. 
If so, don’t expect to find numbers of this kind in the matrices we talk 
about. If you haven't heard about such numbers, just forget we mentioned 
them. We shall have very little use for them in this book. 


1.2 Where Matrices Come From 


The matrices used in the social sciences are of two general kinds. Some 
matrices come directly from our observations of things and people. These 
we call data matrices. Other matrices come from data matrices when we 


apply certain rules to them. These we call derived matrices. When we 
apply appropriate rules to some data matri 


ces, we may get extremely 
interesting and useful results, 


Because of this difference of 
me of the mathematician’s rules. 
ient to emphasize some rules the 


price of stock 2 during t 


А good example of matrices that ar 
thousands of people throughout the со 
pages of any daily newspa 


he third month was 13. 


of great interest to hundreds of 


untry may be found on the sports 
per. Look at 
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Fig. 1.2.1 


The five rows in Fig. 1.2.1 represent five different baseball teams and the 
three columns are respectively runs, hits, and errors. 

Another example comes from the weather report in a typical daily 
newspaper. 


94 63 .00 
96 73 .24 
64 49 .12 
81 55 .00 
72 58 .56 
75 60 .00 
Fic. 1.2.2 


In Fig. 1.2.2 each row represents а city in the United States, and the 
columns represent, in order, the highest temperature, the lowest tempera- 
ture, and the rainfall in inches for the day. You see, then, that almost any 
Place you look you can find a data matrix, for a data matrix is basically a 
very simple way of organizing the numbers used to describe the various 
kinds of information that come to our attention every day. 


1.3 What the Rows and Columns Mean 


Rows and columns play an important role in both data and derived 
matrices. You should therefore get to understand at once the simple 
Meanings of rows and columns in data matrices. In the first example in 
Fig. 111 the rows represent four different people, and the columns are 
Brades of these persons in three different subjects. In Fig. 1.1.2 the rows 
Tepresent five different common stocks, and the columns are the highest 
Prices reached in each of four successive months. In Fig. 1.2.1 the rows are 
five different ball teams and the columns are records for each of three 
different things that ball teams do. In Fig. 1.2.2 the rows are six different 
Cities, and the columns are four different things about the weather in those 
Cities on a given day. Ordinarily then, you could say that the rows ina 

ata matrix represent things which exist in and of themselves. These we 
call entities. 

Now in all four cases the columns represent things that do not exist in and 
9f themselves. We can't have a grade in a school subject unless Somebody 
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makes the grade. Neither can we have the highest price in a given month 
unless it is the highest price of some particular thing like a common stock. 
In the same way, we can't have a hit or a run unless someone makes that 
hit or run. Nor can we have a temperature reading or rainfall except at 
some particular place. We may say then that usually the columns of a 
primary matrix represent attributes of the entities in the rows. These 
attributes are sometimes called variables. 

In general then, we may say that in a data matrix the rows represent 
entities and the columns represent attributes. We could of course let rows 
be attributes and columns be entities. The plan we have adopted is fairly 
common practice, however, because usually there are many more entities 
than attributes in a matrix. The weather report is an example. Usually 
the number of cities on which weather is reported is much greater than the 
number of different things reported about the weather. Similarly, the 
daily stock market report lists hundreds of stocks, but ordinarily only à 
small number of things are reported about each stock—such as the high, 
the low, the close, and the number of shares traded. So also а matrix of 
School grades would usually not include more than six to eight subjects, but 
it might include dozens or even hundreds of students. 

For convenience we have come to construct data matrices with rows as 
entities and columns as attributes. In the width of an ordinary newspaper 
column most of the important attributes about a particular subject can be 
given, but this space would not hold more than a few of the many desired 
entities, Furthermore, we are used to reading horizontally across the page, 
and our interest tends to focus on a Single entity rather than a single 
attribute of a lot of entities. We can, therefore, by following our natural 
reading habits, find all that is recorded about a single entity in one row. 

To summarize what we have been saying, 
includes more entities than attributes ; this is on 
let the rows represent entities. 


a data matrix ordinarily 
e reason why we choose to 
Furthermore, to be useful for scientific 
purposes, a data matrix must in general have more entities than attributes. 

You should know, however, that the method we use here is by no means 
Standard practice among scientists. Some writers prefer to let entities be 
columns and attributes be rows. Others are not consistent; they use both 
styles of matrix construction. In this book, however, unless we give warning 
to the contrary, we shall stick to the rule of using rows for entities and 


columns for attributes in a data matrix, 
1.4 Changing the Order of Rows or Columns 


One important feature of both entities and attributes is that the order in 
which they are recorded i 


INTRODUCTION а 


1.1.1, which row comes first, second, and so on, nor in what order the 
columns are arranged. For practical purposes it may be convenient to have 
persons or cities or other entities listed in alphabetical order, but this is not 
necessary as long as it is clear which row is which person or city. In the 
same way, in Fig. 1.1.2 it might be convenient to have the attributes, the 
school subjects, in alphabetical order as Algebra, English, History, and so 
on, but this is not necessary as long as it is clear what each column is. You 
can state this principle by saying that rows or columns of a data matrix 
usually do not represent a scale or continuum (plural continua). 

An example of a matrix which is not a data matrix and in which neither 
the rows nor the columns are interchangeable is Fig. 1.4.1. 


Weight in pounds 


3 
4 
3 


Height in inches 
n 


10 


Fic. 1.4.1 


This table represents а group of ten boys who range in height from 61 
through 63 inches and in weight from 90 through 92 pounds. Each number 
їп the table is the number of boys in the group having а specified weight 
and height. For example, the number of boys who are 62 inches tall and 
Weigh 91 pounds is three. The number of boys who are 61 inches tall and 
Weigh 92 pounds is only one. But Fig. 1.4.1 is not a data matrix. The rows 
represent a scale or continuum of height and the columns represent a 
Scale or continuum of pounds. You can easily see, however, what kind 
of а data matrix Fig. 1.4.1 would come from. Since all the numbers 
In the table add up to 10, there were ten boys altogether. Therefore, the 
entities in the data matrix were boys, and it had ten rows. The matrix had 
two columns or attributes, one of which was height and the other weight. 


1-5 Some Exceptions to Interchangeability of Rows or Columns 


There can be exceptions to the rule that the order of entities or attributes 
of data matrices can be changed about. The most important is a matrix in 
Which either the entity or the attribute is some sort of time sequence, such 
as days or months. In Fig. 1.1.2, for example, the entities are common 
Stocks and the attributes are successive months. While it would be quite 
Possible to put July before June or May before April, it would be hard to 


ur of а good reason for doing so. Again you might have a matrix as in 
.L52, 
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20 30 33 
10> 35 21 
19 19 22 
20 21 22 


Fia. 1.5.2 


where the rows are students in a typing class and the columns are the num- 
ber of words per minute in each of three successive weeks. You would not 
want to put the first week last or the second week first. 

Similarly, the entities might be successive time intervals. This could 

happen if all the numbers for the matrix were obtained from a single 
person. Suppose, for example, this person were ill in a hospital and each 
day for two weeks a record was made on each of three attributes—his 
temperature, blood pressure, and pulse rate. Then you would have a 
matrix of 14 rows and 3 columns. Each entity, now, would be the individual 
on each of the 14 days. Here again you would not ordinarily switch the 
days out of their natural Sequences. 
A really thorough treatment of the cases in which time sequences can be 
ther entities or attributes would lead us into problems that need not 
concern us in this book. Therefore, you may simply remember that for the 
most common data mairices the order of entities can be changed about in 
апу way you please and so also can the order of the attributes. 


ei 


1.6 Reciprocal Nature of Entities and Attributes 
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1.7 Scientific Predictions and Data Matrices 


But what is a matrix used for? In the social sciences, it is a basis for 
prediction. For a partieular group of entities, some attributes may be 
predicted from others. All scientific predictions are based on the numbers 
in data matrices and on certain operations or rules applied to them. "This 
is true whether you are making scientific predictions about the weather, 
What grades you will make if you major in English, how successful you will 
be as a doctor, how happy your marriage will be, how much oil, if any, can 
be found on Uncle Ben's old wheat farm, or the price of General Motors 
common stock by next Christmas. If the predictions of these or any other 
events you can think of are not based directly or indirectly on data matrices 
and operations upon them, then you can be sure that the predictions are not 
Scientific, 

Ordinarily, we wish to predict certain attributes for specific entities. If 
your future is in question you are the entity, and the attributes are your 
English grades, success as a doctor, or marital happiness. Similarly, in our 
other examples, Uncle Ben's wheat farm is the entity, and the attribute is 
the amount of oil it can yield; and the entity is General Motors common 
Stock, and the attribute is its price next Christmas. 


1.8 The Criterion Attributes 


Usually the attributes you want to predict are the kinds of things that 
mean a great deal to you or someone else. Or, they may be things which 
Over a period of time would turn out to be of some importance to a great 
many people, It is important to you to know whether you will be a success- 

ul doctor or whether your marriage will be a happy one. 

Another characteristic of the kinds of attributes we wish to predict is 
that finding out about them directly could be very costly. Assuming you 
had the money and could get through medical school, you could try 
Practicing medicine and see how you made out. But if you failed, think of 
all the money, time, and effort wasted in preparing for a medical career. 

The third and most important characteristic of the kind of attributes we 
Wish to predict is the very fact that the measure of that attribute for a 
Particular entity is not available at present. A prediction necessarily deals 
with the future, 

We call the attribute we wish to predict the criterion attribute or simply 

€ criterion (plural criteria). 

20% in any scientific prediction study, you may well have two or more 
criteria, You may want to predict your grades not only in English, but in 
mathematics, history, and many other subjects, or you may want to predict 
Hot only how successful you would be as a doctor, but also as an engineer, 
an architect, a lawyer, and so on. 
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1.9 The Predictor Attributes 


In a general way you can say, then, that for Scientific prediction the 
columns or attributes in a data matrix are of two kinds—those we want 
to predict, or the criteria, and those we use to make our predictions. These 
latter we call predictor attributes or variables, In general we are able to 
find numbers or measures of predictor attributes for each of a group of 
entities, such as people, sooner and more economically than we can get the 
numbers for the criterion attributes or columns. See Fig. 1.9.1, for example. 


21 63 
22 66 
23 69 
24 72 
25 75 
26 78 


Fig. 1.9.1 


Here we have a matrix of Six rows and two columns. Suppose that each row 
represents а bookkeeper in a big department store. 
first column represents the 
arithmetie test before they w 
column represents their weekl 


attribute. So also, if you are 


is related to competence in a bookkeeper, you see that the first column 
might be a predictor attribute, 


1.10 The Prediction Formula 


Now take а £o 
second column is three times the 
draw the conclusion that if you К 
test before employment, 
bookkeeper in the depa; 
course, you could never 
by knowing his score о 
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the sort given in Fi 
abl он given in Fig. 1.9.1 have shown that for some jobs a test is a reason- 
ably good indication of later success. 

Now look at Fig. 1.10.1. 


40 26 184 
28 21 140 
60 23 212 
43 22 174 
21 25 142 
30 28 172 


Fig. 1.10.1 


Suppose that the rows represent secretaries in a large business office, and 
that the last column represents their monthly salary at the end of the first 
year of employment. This column will then be the criterion attribute. 
Suppose the first column gives the scores on an arithmetic test taken before 
the girls were employed, and the second their scores on a reading test. Now 
if you were good at spotting number relationships, a little study of the 
figure would reveal to you a rule that relates the first two columns to the 
third. You could also find this relationship, however, by using some simple 
rules of matrix algebra. If you applied these rules you would find that if 
you took two times the arithmetic score of a secretary plus four times her 


reading score, you would get her monthly salary after one year of employ- 


ment, Here again, we have greatly oversimplified the matter. It is true, 
better estimate of how successful 


however, that in many cases we can get à 
a person will be on a job if we have several test scores. 

You see now that the essential characteristic of the predictor attributes 
is that they enable you to reconstruct the criterion column. By using the 
simple rules of matrix algebra, we find how accurately it is possible to 
reconstruct the criterion column from the predictor columns, and the rules 
show us exactly how to do it. For example, in Fig. 1.10.1 the application of 
the approximate rules would show that two times the number in the first 
column plus four times the number in the second column would give the 
corresponding number in the third or criterion column. | © 

Thus matrix algebra is helpful when you deal with a matrix of entities 
and attributes in which one column is а criterion attribute. It helps you 
to find the numbers to use in multiplying the numbers in each of the other 
columns, so that when you add the products together for a given entity you 
get as close to the corresponding number in the criterion column as possible. 
The numbers you multiply by are called weights. For any one attribute, 
you will always use the same weight for all the entities. For example, in 
Fig. 1.10.1 if you multiply the arithmetic score by two and the reading 
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score by four for one person, then you must also use two for the arithmetic 
Score of every other person and four for his reading score. 

Sometimes matrix algebra will help you discover that no matter what 
weights you use, it is not possible to reconstruct the numbers for the 
criterion attribute at all closely. In any case, however, you can find out how 


closely it is possible to reconstruct them—exactly, not at all, or somewhere 
in between. 


1.11 The Bases of Scientific Prediction 


But what is the good of reconstructing measures of criterion attributes 
for different persons or entities when you already know them? 'The truth 
is that, for these particular persons or entities, reconstructing the criterion 
attributes does no good at all. Here we must pause to state a principle that 
is basic, not only to all Science, but to all knowledge. The only way to 
judge what will happen in the future is by what has happened in the past. 
Other things being equal, the more frequently things have happened in the 


past, the more sure you can be they will happen in the future. That is, the 


criterion attributes are a tool of the empirical approach to the search for 
truth. 


Your only purpose in examining a matrix of persons or entities whose 
criterion attributes you already know is to get weights to use on the 
predictor attributes of those entities whose criterion attributes you don't 
know, in order to predict them. This is one of the reasons why a data 


matrix generally has many more entities than attributes. The entities 
represent experience, and the 


more experience is represented i 


the statistician's “degrees of freedom.” This subject is treated in detail in 
Statistics texts. 


1.12 Two Kinds of Entities 


We may then call a data matrix with b ; 2 
‘ oth predict, rion 
attributes or variables a zug predictor and criterio 
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40 2 184 
28 21 140 
Ї 60 23 212 II 


43 22 174 
21 25 142 
30 28 172 


32 22 a 
III 37 29 x5 IV 
56 20 
Fig. 1.12.1 


You may very appropriately call Fig. 1.12.1 a complete prediction data 
Matrix. Figure 1.10.1 you may call an experimental prediction data 
matrix, because it includes only the entities used to determine experimen- 
tally the weights to be applied to the predictor attributes of the entities 
whose criterion attributes are unknown. 

Therefore, we call the entities whose criterion attributes are known 
experimental entities, and those whose criterion attributes are to be de- 
termined administrative entities: 

Returning to Fig. 1.12.1, then, you see that a complete prediction data 
Matrix consists of four sections or quadrants. Quadrant I, the upper left, 
Consists of the predictor attributes for the experimental entities. Quadrant 

] the upper right, consists of the criterion attribute for the experimental 
entities, Quadrant III is the predictor attributes for the administrative 
entities, Quadrant IV must be left blank until you have applied to the 
numbers in the other three quadrants the appropriate rules of matrix 
algebra that will enable you to get a prediction of the numbers in this 
quadrant, 

You should also notice in passing that in Fig. 1.12.1 there are blanks at 
the bottom of Quadrants III and IV. These blanks indicate that usually 
m à complete prediction matrix the number of administrative entities is not 
definitely known. Once you have used the experimental part of the matrix 
to determine the appropriate prediction weights, you may go on applying 

ese weights more or less indefinitely to the predictor attributes of one 
administrative entity after another in order to predict their criterion score 
Ог attribute. Thus, once you found the weights to be applied to arithmetic 
and reading scores for predicting success as a secretary, you might go on 
enr after year using these weights to predict the success of each applicant 
Or the job, 

On the other hand, the number of entities in the experimental part of the 


Matrix must be fixed for any particular research or study. Once you start 
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operating on it with the rules of matrix algebra, you can't very conveniently 
add or take out rows. You should be sure that you have in it the number 
and kind of entities that are needed to give results you can have confidence 
in. The same is true with the number of attributes. Whether or not you 
have the right number and kind of each depends on a great many things. 
We can give very little attention to these in this book, but you should 
know several of them. 

The human sciences, such as economics, anthropology, psychology, 
education, and political science, are of course very complex. To predict 
things about people, things that are important to them, requires that you 
take into consideration a great many attributes. "Therefore, other things 
being equal, the more predictor attributes you have in an experimental 
prediction matrix, the more accurately you can expect to predict the 
criterion for the applied entities. 

For best results in prediction, however, the number of entities in the 
experimental matrix should be much greater than the number of attributes. 
The more predictor variables you have, the more entities you will need. 
You can learn how to tell whether you have enough entities and attributes 
in an experimental matrix by reading technical books in statistics. What 
you should know now is that in the human sciences, the experimental 


prediction matrix should be large if the results are to be useful for pre- 
diction. 


SUMMARY 


1. What a matrix is 


а. Definition. A matrix is a table of numbers or symbols that stand for 
numbers, having rows and columns. 
b. Example. 


o 0 - 
чњ № 
bo or оо 


2. Source of matrices: 


a. Data matrices come from experimental observations. 
b. Derived matrices come from operations on data matrices. 


3. Meaning of rows and columns of data matrices: 
a. Rows usually consist of entities, representing people or things. 
b. Ke Ag usually consist of characteristics or attributes of the 
entities, 


4. Interchangeability of rows and columns: 
а. Тһе Serial order of the entities and of the attributes is usually 
irrelevant. 
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b. Entities and attributes do not usually represent continua. 
c. Dimensions of derived matrices may be continua. 


5. Exceptions to interchangeability of rows and columns: 


à. Time intervals for entities are not interchangeable. 
b. Time intervals for attributes are not interchangeable. 


6. Reciprocal nature of entities and attributes: 


10. 


п. 


l. 


à. The experimenter makes the distinction between entities and attri- 
butes. 

b. The rules of matrix algebra do not depend on a distinction between 
entities and attributes. 


- Scientific prediction and data matrices: 


a. All scientific predictions must be based on data matrices. ! 
b. Criterion and predictor attributes are required for scientific predic- 
tion. 


Criterion attributes: ' А 
а. Definition. Criterion attributes are those we wish to predict. 
b. Criterion attributes are important as such, are costly to assess, and 


àre not currently available. A З 
С. А data matrix may have more than one criterion attribute. 


Predictor attributes: , 

a. Definition. Predictor attributes are those we predict from. 

b. Predictor attributes are not necessarily important in themselves, 
аге relatively easy to assess, and are currently available. 


The prediction formulas: К 
а. If predictor attributes are to be useful they must have numerical 


relationships with the criterion attributes or variables. 
b. Matrix algebra enables us to find these relationships. 


Kinds of entities: i i 
a. Definition. Experimental entities are those from which the predic- 


tion formulas are deduced. : 
b. Definition. Administrative entities are those to which the formulas 


are applied. 
EXERCISES 


Make up a data matrix from which Fig. 1.4.1 could be derived. 


Given the data matrix 
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x У 
1 1 
3 5 
2 3 
£T 
8 15 
0 Т 

7 ? 


(a) What should the last two numbers in the Y column be? 
(b) Write an equation for the relationship of the Y to the X column. 


3. Given the matrix 


1 
2 
Б] 


3 
6 
9 


on 


D 


How can this be rewritten by changing only one number so that the rows and 
columns are proportional? 


ANSWERS 
1. Height Weight 
90 61 
90 61 
91 62 
91 62 
91 62 
91 63 
92 61 
92 62 
92 63 
92 63 
2. 
(a) 11, 13 
(b Y 22X —1 
3. 


с t9 — 
с ҥ to 
оо © 


Chapter 2 


The Language of Matríces 


It is quite possible to use the kind of algebra you learned as a high school 
freshman on the data matrices from the behavioral sciences, but since these 
matrices are usually large, elementary algebra is very tedious and confusing 
to use. For the analysis of large tables of numbers, matrix algebra is ideally 
suited. We shall therefore in this book learn how a whole table of numbers 
may be handled with matrix algebra much as you handled single numbers 


with elementary algebra. 


2.1 Showing that a Table of Numbers Is a Matrix 


Methods of designation 

We have said that basically a matrix is a table of numbers; now we need 
to elaborate on this definition. When we call a table of numbers a matrix, 
we imply that it is subject to a set of rules indicating what can be done with 
these numbers, and also what cannot be done. 

There are several ways in which a table of numbers can be designated a 
matrix, One way is to put a double ruling on either side of the table. See 
Fig. 2.1.1. 


4 3 1 
à 3-5 б 
13 4 
T 9 8 
Fic. 2.1.1 


A second method commonly used to show that a table of numbers is a 
matrix is to enclose it in parentheses as in Fig. 2.1.2. 


5 29 172 14 
6 8 i$ 5 
и з 2 10 
IB le 8 18 
14 9 3 I 
Fic. 2.1.2 


17 
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A third method is to enclose the matrix in brackets, as in Fig. 2.1.3. 


43 I 
6 2 5 
1 3 4 
v9 5 
Fig. 2.1.3 


All three of these methods—the double ruling, the parentheses, and the 
brackets—are equally good. In textbooks on matrix algebra, and in 
published articles which use matrix algebra, you will find that sor 
use one method of designation and some use another. 
may use more than one of the methods. 


As you will see later, а single column or a single row of numbers may be 
regarded as a special case of a matrix. Once we start applying the rules of 
matrix algebra, it is important to know when a matrix of this kind should be 


considered as a row and when it should be considered as a column. In order 
to save space in printing, 


column matrices as rows. 
printed should really be a 
as shown in Fig. 2.1.4, 


me writers 
In fact, some writers 


it is sometimes customary to write all single 
Then in order to show that a row matrix as 
column matrix, the row is enclosed in braces 


(43.5 7} 
Fic. 2.1.4 
Here the braces mean that the row sh 


column, as in Fig. 2.1.5, exce 
column. 


ould really have been written as а 
pt that there wasn't Space to write it as a 


4 4 4 

5 ог S or Е 

5 5 5 

7L 7 7 
Fie. 2.1.5 


Now if the one-line matrix in Fig. 2.1.4 should actually have been written 


aS а row, rather than as a column, it would have been written in one of the 
usual three forms shown in Fig. 2.1.6. 


l4 3 5 vl 
(4 3 5 7) 
K 35 7 


Fig. 2.1.6 
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One thing you must carefully avoid in designating a table of numbers as 
à matrix: never use a single line on each side of the table of numbers, as in 
Fig. 2.1.7. 


4 3 1 
0:2: ^9 
MED E: 
Fig. 2.1.7 


If a matrix has the same number of rows as it has columns, as Fig. 2.1.7 does, 
the single line on each side of the table of numbers has a very definite 
meaning to mathematicians. Figure 2.1.7 is not a matrix but a determinant. 
In mathematics, a determinant is a single number and the line on each side 
of the table means that you multiply various combinations of the numbers 
together and then add all of these products together to get that single 
number, In this course we shall not be concerned with how to get a deter- 
minant or single number from a square table of numbers. Although most 
mathematicians and people who use and write about matrix algebra also 
use determinants in matrix algebra, we shall have no use for determinants 
in this book. The use of determinants needlessly complicates many of the 
rules and procedures of matrix algebra. Just remember never to use a single 
ruling on each side of a table of numbers to show that it is a matrix. 


Preferred designation 


In this book we shall use brackets around the numbers to show that the 
tables are matrices. These are more convenient than the double ruling on 
both sides simply because it is difficult for some people to draw two parallel 
vertical lines quickly. For matrices that consist of a single row we shall also 
ordinarily use brackets, as in Fig. 2.1.8. 


[3 7 9] 
Fig. 2.1.8 o 


In general, it is more practical to use brackets for a matrix that has a large 
number of rows. Actually it doesn't matter much what method you use as 
long as you are consistent. In no case, however, shall we use braces to show 
that numbers that have been written in a single row should really have been 
Written in a column. If the matrix is a row, we shall write it as such and if it 
isa column, we shall write it as а column. 


2.2 Specifying the Size of a Matrix 


As you might suppose, the size of a matrix is determined by the number 
of rows and the number of columns it contains. The number of rows and 
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columns is called the order of the matrix. In giving the order of а 
it is customary to mention the number of rows first and the number of 
columns second. For example, Fig. 2.1.1 isa four-by-three matrix because 
it has four rows and three columns. Figure 2.1.2 is a five-by-four matrix 
because it has five rows and four columns. In giving the order of a matrix 
it is customary to write “four by three” as 4 by 3. A five-by-four matrix 
would be written 5 by 4. For example, if we had a matrix of scores on 
10 different tests for 20 different persons, this would be a 20 by 10 matrix. 
The term dimensions of a matrix is used by social sc 
exactly the same thing as order. The word in this use is not 
mathematicians, however. A matrix, as we use the term, 
sions; one of height and one of width. As you would guess, t 
matrix is simply the number of rows it has. The w 
columns. We mention height first and width 
It is natural, then, that if the hei 
we call it а vertical matrix. 
Figs. 2.1.1, 2.1.2, 2.1.8, 2.1 


matrix, 


ientists to mean 
common among 
has two dimen- 
he height of the 
idth is the number of 
second. 

ght of a matrix is greater than its width, 
A vertical matrix has more rows than columns. 
-5, and 2.1.9 are all vertical matrices. Similarly, 
if the width of a matrix is greater than its height, we call it a horizontal 
matrix. A horizontal matrix has more columns than rows. Figure 2.1.8 is 
ап example of a horizontal matrix. Another example is Fig. 2.2.1. 


64 3 
7 29 
Fia. 2.2.1 


Remember, however, that F 
braces show that it should 
that it is therefore a vertical matrix, 


379 
426 
8 1 3 
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Here we have a matrix of three rows and three columns, or a 3 by 3 matrix. 
You ean indicate in several ways that a matrix is square. In giving the 
order you can simply give it as you would for any other matrix. Thus you 
would say a 3 by 3 matrix, a 4 by 4 matrix, and so on. Perhaps the simplest 
and best way to indicate that a matrix is square, if you want to give its size 
at the same time, is simply to say a third-order matrix, a fourth-order 
matrix, and so on. The fact that you give only one number in specifying 


the order means that both dimensions of the matrix are equal to this num- 
ber. 


2.3 The Numbers in a Matrix 


Methods of designation 


Several different names can be used in referring to any single number 
within a matrix. The number is sometimes called a coordinate of the matrix. 
A less technical way of referring to these numbers is simply to call them 
entries, Any particular number in the matrix is called an entry in the 
matrix. A third way of referring to the numbers is to call them elements. 
We shall somewhat arbitrarily choose the word element to refer to a single 
number within the matrix. An element in the matrix of test scores is simply 
the score of a given person on a particular test. 

A specific element of the matrix is designated by referring to the row and 
column in which it is located. As in specifying order, we mention first the 
row and then the column. For example, the element in the third row, 
Second column, of Fig. 2.1.2 is 3. The element in the third row, third 
column, of Fig. 2.2.2 is 3. 


Total number of elements 


The total number of elements in a matrix is obviously its height times its 
Width, or the product of its two dimensions. The total number of elements 
in Fig. 2.1.3, for example, is 4 X 3, or 12. The total number of elements in 
the matrix of test scores is 200. 


The principal diagonal 


In speaking of a square matrix, such as Fig. 2.2.2, we give a special name 
to all elements whose row and column number are the same. There would, 
Of course, be as many of these elements as the order of the matrix. In 
Fig. 2.2.2 there are three such elements. The element in the first TOW, 
first column, is 3, in the second row, second column, it is 2, and in the 
third row, third column, it is 3. These elements form what we call the 
principal diagonal of the matrix. The principal diagonal of a square matrix 
Consists of the elements that have the same number for both the row and 


column. 
З.С.Е R.T., West Benga) 2 & X es We, 
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2.4 Using a Letter to Stand for a Number in a Matrix 


Numerical subscripts of letter elements 


You will recall that in elementary algebra you used letters such as 
а, b, c, z, y, г, and so on to stand for numbers. We use the same practice 
in matrix algebra. In developing the rules for matrix operations, it is 
convenient to let letters stand for the elements of the matrix. The letters 
may do this in various ways. One method is shown in Fig. 2.4.1. 


a а аз 

bi b; bs 

с C C3, 
Fic. 2.4.1 


Here we use different letters to represent the different rows of the matrix. 
The letter a is used to represent, numbers in the first row, b, numbers in the 
Second, and c, numbers in the third. Here we have a third-order square 
matrix. To designate the columns, we use a subscript on each letter. (A 
subscript is a small letter or number written а little below and to the right 
of another number or letter.) The subscripts in Fig. 2.4.1 designate the 
columns in which the elements are located. Thus, if Fig. 2.4.1 represented 
the numbers in Fig. 2.2.2, bs would represent 6 because the element in the 
second row, third column, of Fig. 2. 


2.2 is 6, and the element in the second 
row, third column, of Fig. 2.4.1 is by. 


Another plan you could use to let letters stand for the elements of a 
matrix is similar to that in Fig. 2.4. 1, but the subscript of the letter indicates 
the row and the letter itself indicates the column, as in Fig. 2.4.2. 


а b а 
а b, б 
as bs c 
а b. c 


Here you have a 4 b 


the b’s in the second, and all the c's in the third. Е 


весог n. Figure 2.4.2 might be used to represent the matrix 
in Fig. 2.1.3. Then аз 
Which is the number 1. 


The third way to indicate that a letter stands for a number in a matrix is 
а уе xam letter with two subscripts. This principle is illustrated in 
ig. 2.4.3. 


THE LANGUAGE OF MATRICES 23 


an а авз 
аз аљ аз 
аз аз азз 
аз аә as 


Fic. 2.4.3 


Here we have used the letter a to stand for each element in the matrix, but 
a different pair of subscripts gives the location of each element. The first 
number of the subscript is the number of the row in which the element is 
found. The second is the number of the column in which the element is 
found. Thus, here a; is the element in the second row, third column. You 
will see that the first number in each element is the same in any given row. 
The second number of the subscript is the same for every column. Without 
looking at Fig. 2.4.3, then, you should be able to tell what as; is in Fig. 2.1.3. 
It is the element in the second row, third column, or 5. 

In this book, we shall use this third method to let a letter stand for an 
element of the matrix. It is more convenient than the methods shown in 
Figs. 2.4.1 and 2.4.2, because as soon as the subscripts of an element are 
given, you know at once in which row and which column to find it. On the 
other hand, if you use the methods of either Fig. 2.4.1 or Fig. 2.4.2, you first 
have to remember which of the two you decided to use, so as to know 
whether the letters are rows and the subscripts columns, or the other way 
around. Even assuming you can remember this, if you use more than three 
or four letters, you might have trouble deciding promptly whether f, say, 
is the fifth, sixth, or seventh letter in the alphabet. 

You must note, however, that although in general we shall use the double 
subscript system, where the first subscript means row and the second means 
column, there will be some exceptions when the order of the subscripts will 
be reversed. We shall call your attention to these exceptions when the 
Occasions arise. 


Letter subscripts of letter elements 


You have seen how it is convenient to let letters stand for the elements of 
a matrix and to use subscript numbers to indicate in which row and column 
cach element is located. Sometimes it is also convenient to use letters for 
the subscripts, as in Fig. 2.4.4. 


Qu G2 ... Am 
ааз аә ... Am 
Ani An ... Uam 


Fia. 2.4.4 
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If the matrix has a large number of rows and columns, as many data 
matrices do, it might be far too much trouble to write out all of the letter 
elements with their appropriate number subscripts. Therefore we write 
the first two elements in the first row and then use a series of three dots 
(ellipsis) to indicate that all of the other elements in the first row except the 
last are taken for granted. We write the last element aS ам, Where m stands 
for the number of columns. Here we do not commit ourselves as to what 
m is. We can let it be anything we please to suit any particular matrix. 
In the same way, the second row contains the first two elements, then three 
dots, and finally, the elements 4», to indicate that this is the last of the 
m elements in the row. Then we have а row of dots the full width of the 


matrix to show that there are a number of other rows that we simply don't 
take the time to write in. 


Finally, we write in the first two elements of the 1 
three dots, and the last element. 


in the last row has n as a first sub: 


ast row, followed by 
You will notice that each of these elements 
script. This means that we do not commit 
ourselves specifically as to the number of rows in the matrix. We let n stand 


for the number of rows, which can be anything we please. You see then 
that the last element in the last row 


The first one indicates the number o 
the number of columns. 


numbers. 


A system closely resembling that in Fig. 2.4.4 is given in Fig. 2.4.5, 


Ши. жи ш 
Qni ... ny 
Fig. 2.4.5 


Here we give only four elements in th 
row, the last element in the first row, 
the last element in the last row. Е 
Figure 2.4.4, but some people find 
in à row mean if the second eleme 


e matrix: the first element in the first 
the first element in the last row, and 
igure 2.4.5 is a little more compact than 
it easier to remember just what the dots 
nt in each row, and the second row, are 


À third variation of the use of letter Subscripts is shown in F ig. 2.4.6. 


а а; ањ 
ад Qij а 
uh v Uy fv Vas. 


Fic. 2.4.6 
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Here we give three rows of the matrix and three elements in each row. 
The method is similar to Fig. 2.4.5 except that in row 1 we put an element 
between the first and the last one, with the second subscript given as j. 
We also put a row between the first and last row, with č as the first subscript. 
We mean that this is the ith row, or any unspecified row between the first 
and last one. In the same way, the column between the first and last one 
means that this is the jth column, or any unspecified column between the 
first and last one. The intermediate element in the ith row, then, is aj;. 
We can then refer to the ijth element of the matrix, without specifying 
which particular row or column the element is in, so as to make general 
statements about the matrix elements. 

In all three examples given in Figs. 2.4.4, 2.4.5, and 2.4.6, you will see that 
Whenever a letter is used for a subscript it has the same meaning as a num- 
ber. When the letter is in the first position, it refers to the row in which the 
element is found; in the second position, it refers to the column. In all three 
methods, the subscripts of the lower right element indicate the dimensions 
of the matrix. Thus, in each of the three examples, the element anm means 
that we have an n X m matrix, or a matrix with n rows and m columns. 


2.5 Using a Single Letter to Stand for an Entire Matrix 


What makes matrix algebra so useful in working with large tables of 
numbers is that we can let a single letter stand for an entire matrix. Figure 


2.5.1 is an example. 


ап Q2 ... аһ 
аз аљ ... 02m 
а= 
аһ Anz ... Qnm, 
Fra. 2.5.1 


Here we let the single letter a without any subscripts represent'an entire 
Matrix whose elements are represented by the letter a with subscripts. 


Another example is Fig. 2.5.2. 


bu ana bis 

Beate saa Dy 
Dai ee Dam 
Fic. 2.5.2 


Here the letter b stands for a matrix whose elements are represented by 
the letter b with subscripts. You can choose any letter you wish to repre- 
Sent a matrix. In the two examples, we use the same letter to stand for the 
Matrix and for the individual elements in the matrix. This is not necessary, 
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however, for we can use a single letter to represent a matrix whose elements 

: av 
are numbers. We can use the letter c, for example, to stand for the matrix 
given in Fig. 2.2.2. This we would write as in 


379 
с=|4 2 6 
8 1 3 

Fig. 2.5.3 


Neither is it necessary that we use the same letter to stand for the matrix 


even if the elements are letters. We could, for example, write a matrix as in 
Fig. 2.5.4. 


Cu Cm Сз 

Cor Сә боз 
pel 

C31 C32 Cag 

Ca Са Ca, 


Frc. 2.5.4 


Here we let F stand for a matrix w 
letter c. There is nothing in the rul 
select a single letter to stand for an 
made on the basis of simplicity or c 
you would doubtless prefer to use t 
letter that stands for the entire ma 
for the elements in the matrix. 


hose elements are represented by the 
es of matrix algebra to tell us how to 
entire matrix. Such decisions must be 
onvenience. Other things being equal, 
he plan of Figs. 2.5.1 or 2.5.2, where the 
trix is the same as the letter that stands 


Using a single letter to stand for a number greatly simplifies algebraic 
manipulations and also the description of the computation worksheets 
required for the analysis of a particular set of data. 


Specificution of order 


ed to write down the 
examples. You could simply say, 


with n and m standing for any unspecified but different numbers. That is 
enough to define the order, 


matrix algebra which you 
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Another method used to indicate the order of a matrix is to use subscripts 
on the letter which stands for the entire matrix. If you do this, you must 
state clearly, however, that the subscripts indicate the number of rows and 
columns in the matrix and that the letter stands for the entire matrix. 
(Subscripts may be used for various other purposes.) If you adopt this 
practice, the first subscript will ordinarily represent the number of rows 
and the second letter the number of columns. For example, you could say, 
"Let anm be an n by m matrix." It is perfectly clear then that the symbol 
anm refers, not to the element in the nth row and mth column, but to an 
entire matrix with n rows and m columns. Usually when subscripts are 
used to indicate the order of a matrix, they will be letters rather than 
numbers, This, however, is not an absolute rule and we may occasionally 
make exception to it. The use of letter subscripts to indicate the order of a 
matrix is of value when we take up supermatrices in Chapter 5, and matrix 
multiplication in Chapter 9. 


SUMMARY 


1. Showing that a table of numbers is a matrix: 

а. If a table of numbers is designated as a matrix, it is subject to a well 
defined set of operations. 

b. A table of numbers may be designated as a matrix by enclosing it in 
double rulings on each side, parentheses, or brackets. Braces may 
be used to enclose a row of numbers that is meant to be а column. 

с. The preferred designation in this book is the use of brackets. 


- How to specify the size of a matrix: 
a. The number of rows and columns in a matrix is called its order. The 
number of rows is usually mentioned first. 
b. By dimensions we mean the same thing as order. The height is the 
number of rows and the width the number of columns. A vertical 
matrix has more rows than columns. A horizontal matrix has more 


columns than rows. 


- The numbers in a matrix: 

a. A single number in a matrix is called an element of the matrix. 

b. It may also be called a coordinate or an entry. 

€. When a particular element in the matrix is specified, its row position 
is usually mentioned first and its column second. 

d. The number of elements in a matrix is the product of its height and 
width. 

€. The principal diagonal of a square matrix consists of the elements 
whose row and column positions are the same. 


4. Using a letter to stand for a number: 
a. A letter with double numerical subscripts may be used to indicate the 


= 


ы 


Ae 0 
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elements in the matrix. The first number in the subscript generally 
indicates the element’s row position and the second its column. 
b. A letter element may have letter subscripts to designate any row and 
column location. 
. Using a single letter to stand for a matrix: | 
а. A single letter used to stand for an entire matrix simplifies algebraic 
manipulations and the description of computation worksheets. 
b. The order of a matrix represented by a single letter may be indicated 
verbally, as, for example, n by m, (3 by 4), etc. 
‚ A single letter standing for a matrix may carry double subscripts to 
indicate its order; usually the first subscript is its height and the 


second its width. 
EXERCISES 
. Which of the following is not a matrix? 
23 4 23 4 2 3 4 2 
ВБИ 35 1 3 1 3 
267 2 Ж 2 7 


2 
A B 


Су с› сл © 
Cocco 
ын № 


‚ What are the heights of matrices whose orders are as follows: 
(а) 4 by 3 (b) 7 by 10 (c) 20 by 4 (d) (n by m) 
- How many elements do each of the matrices in 2 above have? 


- Which of the following matrices have principal diagonals? 


К 23 1 7 LTS 4 2 3 
2 92 3 1 2 6 2569 8 1 
qos 3 7 4. 8 29 


a nen 
= 


. Given’ 


< 

Il 
c» фо oNN 
юк сл о 
м о 0 
ы № № с 


8 4 10 


What are the numerical values of the following elements: bis, bai, bsa, Dao, bai? 
. Given 


Anm, 


What is the height and width of a? 
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. A, is a matrix. Without further information what would you take to be its 


height and width? 


ANSWERS 
1-6 
2. (а) 4 (b) 7 (c) 20 (d) n 
3. а)12 (b) 70 (0) 80 (9) xm 
4. Cand E 
5.3,2,4,1,8 


. Height 4, Width 3 


Height r, Width s 


Chapter 3 


Kínds of Matríces 


There are various kinds of matrices that 


are particularly important in 
the behavioral sciences. 


We shall list them for ready reference. 
3.1 The General Rectangular Matrix 


You are already familiar with the rectangular matrix, in which the height 
and the width, or the number of rows and the number of columns, is 
generally not the same. As you have seen, most of the data we get from 
life situations are of the kind that can be arranged in the form of a rectan- 
gular matrix. Also, most matrices, to be useful for scientific prediction, 
must have more rows than columns, or more entities than attributes. 
While most data matrices are rectangular, it does not follow that all 
rectangular matrices are necessarily data matrices, Many derived matrices, 


that is, matrices obtained by applying the rules of matrix algebra to data 
matrices, may also be rectangular. 


3.2 The Square Matrix 


Usually we think of a rectangle 
other. You have seen, however, th 
gular mattrix in which the height 
Square matrix, one having the sa. 
3.2.1 two square matrices are sh 
the other four rows and columns 


as being larger in one direction than the 
at we can have a special case of a rectan- 
and the width are the same. This is a 
me number of rows as columns, In Fig. 
own; one has two rows and columns and 


2 3 ^4 1 7 
MEA $9 3 4 1 
6 Т А 9 
4 3.2 8 
Fig. 3.2.1 
If we use letters to stand for the elements of a Square matrix and double 
numbe 
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the last subscript for all elements in the last column will be the same as 
the first subscript for all elements in the last row. Figure 3.2.2 is an example 
of a square matrix of this kind. 


ап Gi аз 
an а» аз 
O3 Q32 аз; 


Fig. 3.2.2 


You see that, in this third-order square matrix, 3 is the last subscript for 
each element in the last column, and 3 is also the first subscript for each 
element in the last row. Notice that in a square matrix both subscripts of 
the lower right-hand element are equal. In this case the subscripts are 
both 3. Another example of a square matrix with letter elements is given 
in Fig. 3.2.3. 


а QAR ... Qm 

ал аљ ... an 

Qni аһ ... Ann 
Fig. 3.2.3 


Here the last subscript for each element in the last column is n, and the first 
Subscript of each element in the last row is also n. For the last, or lower 
right-hand element, both subscripts are n. 

As you saw in Chapter 2, Section 2, if we let a single letter such as a 
represent the entire square matrix, we can indicate that the matrix is square 
by saying that a is an nth-order square matrix, or simply, a is an nth-order 
matrix. If we decide to let subscripts indicate the order of the matrix, we 
Would let а„„ be a square matrix of order n. We must be sure to state this 
definition or the symbol might be mistaken for the last element in the last 
Tow of a square matrix. 


3.3 Symmetric Square Matrices 


A special kind of square matrix that is useful in the analysis of data in the 
ehavioral sciences is the symmetric matrix. This is usually a derived rather 
than a data matrix. A symmetric matrix may be defined as one in which the 
Corresponding elements of corresponding rows and columns are equal. It 
15 shown in Fig. 3.3.1. 


i 4 3 
4 15 10 
3 40. I7 


Fig. 3.3.1 
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Look at row 1 and the corresponding column, which is column 1. The 
second element in row 1 is 4 and the corresponding, or second, element in 
column 1 is also 4. In column 2, you see that the elements are, in order, 


4, 15, and 10. In the corresponding row, that is, the second TOW, you see 
that the elements are also 4, 15, and 10. 


Subscript designation 


If you choose to indicate a symmetric matrix by using a letter for the 


element, and double subscripts to give its position, you may do so in several 
ways. Fig. 3.3.2 shows one way. 


аа аг аз 
а» аљ аз 
аз аз аз 


Fic. 3.3.2 


Notice here that the subscripts for the element in the second row, first 
column, are in the same order as the subscripts for the element in the first 
Tow, second column. In both cases, the subscripts are, in order, 1, 2: The 
rule for writing these subscripts is simple. First write the elements with 
their subscripts in the principal diagonal. Then write the elements with 
their subscripts as you would normally, above and to the right of the 
principal diagonal. Finally write in the subscripts for the elements below 


This method of writing subscripts is one of the exceptions to the rule 
given in Section 2.4, that the first subscript indicates the row and the second 
subscript indicates the column. 

It is common practice to use the system of subscripts shown in Fig. 3.3.2 
to indicate that a matrix is symmetric. We may, however, stick to the 
original rule, as in Fig. 3.2.3. Then, to show that the matrix is symmetric, 
we say that а;; = aji This means that the element in the ith row and the 
jth column is the same as the element in the jth row and the ith column. 
For example, in Fig. 3.3.2 the element in the first row, third column, is the 
same as the element in the third row, first column. 

А very satisfactory method of letting a single letter Stand for a symmetric 


matrix is simply to say, for example, let x be a symmetric matrix. Since a 
symmetric matrix is clear 


y defined, there would be no question as to what is 
meant by this statement. 


Writing the symmetric matrix 
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may be followed; the numbers below the principal diagonal may be omitted. 
Even though this practice is frequent, it is usually not recommended. First, 
it is much more convenient in practical applications to have all of the num- 
bers in the table written in. Second, if the numbers are not written in, they 
may reasonably be assumed to Бе 0. As we shall see, there is a special kind 
of matrix in which the numbers, either above or below the principal diag- 
onal, are 0. It would be a serious mistake to take a symmetric matrix for a 
matrix of this type. 

A convenient way to write down a symmetric matrix of numbers is first to 
write the numbers in the principal diagonal and those above and to the 
right. (These numbers are often called the supradiagonal elements.) You 
can then fill in the matrix below the principal diagonal by starting with the 
first column and copying, in order down the column, each number in the 
first row. Then you go to the second column, and beginning immediately 
below the second diagonal, you copy in order each of the numbers in the row 
to the right of the second diagonal. (Numbers below the diagonal are 
termed infradiagonal elements.) In this way you complete each of the 
unfinished columns in the matrix so that it will, in fact, be a symmetric 
matrix, as you can well verify. 

The most common examples of symmetric matrices used in the anaylsis 
of data from the behavioral sciences are the statistician’s correlation and 
Covariance matrices. The nondiagonal clements in the symmetric matrix 
are correlation coefficients or covariances. If you have had a course in 
Statistics, you are already familiar with these terms. You will learn more 
about them in Chapters 12 and 13. 


3.4 The Diagonal Symmetric Matrix 


You have seen that the square matrix is a special case of the rectangular 
matrix, and the symmetric matrix is a special case of the square matrix. 

here are also special cases of the symmetric matrix. The most géneral of 
these is the diagonal matrix. Ina diagonal matrix, all elements except those 
in the principal diagonal are 0 (Fig. 3.4.1). 


Ib 0.0 
0 12 0 
0 Ө 9 
Fig. 3.4.1 


Another way to define a diagonal matrix is to say that every element is 0 
except those whose row and column numbers are equal. In Fig. 3.4.1 only 
the elements in the first row, first column; in the second row, second column: 
and in the third row, third column, are different from 0. 
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Methods of writing the diagonal matrix 


You may quite properly write a diagonal matrix with no entries whatever 
in the nondiagonal positions, as in Fig. 3.4.2. 


5 
12 
9 


Fic. 3.4.2 


If à matrix is written in the form given in Fig. 3.4.2, all elements in the 
nondiagonal positions are understood to be zero. 


We may write a diagonal matrix with letter el 


ements in the same way, as 
in Fig. 3.4.3. 


ац 
аз 


ann 


Fie. 3.4.3 


This is a diagonal matrix of order ^. You see that for each element the 
first and second subscripts are the same. 


It is not necessary, however, to use double subscripts when you let 
letters stand for the elements in a diagonal matrix. You may use single 


subscripts as in Fig. 3.4.4. 
а 
[^ 


аһ 


Fig. 3.4.4 


It is perfectly clear that Fig. 3.4.4 is a diagonal matrix in which a; is the 
first diagonal element, a; the Second, and so on. 

À common example of a diagonal matrix used frequently in the analysis 
of data from the human sciences has for its elements the standard deviations 
of the attributes, These are indicated by the small Greek letter sigma, c. 
A diagonal matrix made up of standard deviations is given in Fig. 3.4.5. 


[A 
92 
T3, 


Fig. 3.4.5 
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Single letter designation of a diagonal matrix 


We may let a single letter stand for a diagonal matrix as well as any other 
kind of matrix. But, since the diagonal matrix has so many interesting and 
useful properties, and since it is used so frequently in the analysis of data 
in the behavioral sciences, a special, easily recognized symbol for it is 
useful. Therefore, we usually let some form of the letter d represent a 
diagonal matrix. The following symbols are commonly used: D, d, A, à. 

Various subscripts serve to indicate the different diagonal matrices we 
are working with. In Fig. 3.4.6, 3.4.7, and 3.4.8 subscripts are used on the 
letter D to indicate various diagonal matrices. 


а\ 
a 
Da = 
Qn 
Fic. 3.4.6 
bi 
Dy = b, 
bs, 
Fic. 3.4.7 
су 
[E] 
D, = as 
04, 
Fia. 3.4.8 


Here Fig. 3.4.6 is an nth-order diagonal matrix, whose diagonal elements 
are а, а, etc. In Fig. 3.4.7 we let D, equal a diagonal matrix whose 
diagonal elements are bi, b», and bs. The subscript ø to D in Fig. 3.4.8 
Indicates a diagonal matrix each of whose elements is a standard deviation. 
Such matrices are useful in deriving correlation matrices from covariance 
matrices, 


3.5 Scalar Diagonal Matrices 


A special case of a diagonal matrix is the scalar matriz, a diagonal matrix 
All of whose diagonal elements are equal (Fig. 3.5.1). 


зоо 
озо 
ооз 


Fig. 3.5.1 
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Here every diagonal element is 3. We can also use letter elements to 
indicate a scalar matrix. Then we omit the subscripts on the diagonal 
elements to show that they are all equal. In Fig. 3.5.2 letters are used to 
designate the elements of a scalar matrix. 


а 0 0 
0 a0 
D. d @ 


Fic. 3.5.2 


3.6 The Identity Matrix 


A special case of a scalar matrix is the identity matrix. The identity 
matrix is a scalar matrix all of whose diagonal elements are 1. Since it is 
still a special case of a diagonal matrix, all of the nondiagonal elements 
are 0. Figure 3.6.1 is an example of a third-order identity matrix. 


15 10.0 
0,0 
бї 


Fic. 3.6.1 


When a single letter is to stand for an identity matrix, we may use the 
capital letter 7. In this book, J always stands for an identity matrix. Not 
everyone uses the capital letter J for the identity matrix; sometimes the 
number 1 is used to represent it. The number 1 has а more convenient use, 
however, for another kind of matrix, which will be considered soon. There- 


fore we shall not use the number 1 for the identity matrix, even though some 
writers do. 


The identity matrix serves the s 
number 1 does in ordinary algebra, 
Scalar algebra is simply the algebr: 
than tables of numbers, 

You can indicate the order of 
is simply to state the order by 
matrix. Sometimes, in the equati 
order by the position in which the identity matrix appears. In such a case 
it is not necessary to specify the order. It is also sometimes convenient to 
'ript. For example, J; would be a 


represent Fig. 3.6.1 then by I5; or, 
rder identity matrix. 


àme purposes in matrix algebra as the 
or what we shall now call scalar algebra. 
a that deals with single numbers rather 


third-order identity matrix, You could 
in general, you could let Г, be an nth-o 
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3.7 The Sign Matrix 


The sign matrix is of partieular importance in research on personality 
traits. It makes possible designation of an attribute, such as a personality 
trait, in terms of its opposite. Thus the plus number may mean introversion 
апа the minus number extroversion. 

The sign matrix is similar to the identity matrix, except that some of the 
l’s in the diagonal may carry minus signs. A sign matrix is shown in Fig. 
3.7.1. 


-1 0 0 
0 1 0 
0/0: el 
Fig. 3.7.1 


The sign matrix is seldom treated as such in standard mathematical books 
and articles on matrices. Nevertheless, because it is useful in analyzing 
data from the behavioral sciences, we give it this special name, using the 
small letter to represent it. A subscript can be used to indicate the order 
of the sign matrix as well as the identity matrix. Unless otherwise specified, 
however, we shall state the order in words and reserve subscripts to indicate 
the particular sign matrix we are talking about. Figure 3.7.2, for example, 
Uses т, to indicate a second-order sign matrix whose first diagonal element 
18 —1 and whose second is 1. 


Figure 3.7.3, on the other hand, uses b as a subscript, so that ù is a matrix 
whose first diagonal element is 1 and whose next two diagonal elements are 


=l 


Fig. 3.7.8 


Note that we have not used 0’s in the nondiagonal positions in either 
Vig. 3.7.2 or 3.7.3. It is not necessary to use 0’s either for sign matrices 


en identity matrices, even though we did use them for clarity in Fig 
2.6.1. > 
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3.8 Skew Symmetric Matrices 


Another special kind of square matrix is the skew symmetric matrix. dn 
a skew symmetric matrix, the principal diagonal is 0, and corresponding 
elements above and below it are of equal absolute value but are opposite 
in sign. Another way of saying this is that aij is equal to —a;;. 


An example 
of a skew symmetric matrix is Fig. 3.8.1. 


0 -3 4 

3 0 —6 

= 6 0 
Fic. 3.8.1 


Here you see that the element in the first. row, second column, is —3, and 
the element in the second row, first column, is plus 3. The element in the 
first row, third column, is 4, and the element in the third row, first column, 


is —4. The skew symmetric matrix plays an important role in certain 
psychological and Sociological scaling techniques. 


3.9 Triangular Matrices 


Upper triangular matrix 


A derived matrix that is useful in the behavioral sciences is a square 
matrix in which all elements below the principal diagonal are equal to 0. 
This is an upper triangular matrix (Fig. 3.9.1). 


257 2 5 

036 or 3 06 

U.-0 T 7 
Fig. 3.9.1 


Note that in the left-hand matrix of Fig. 3.9.1, 0’s are written in below the 


principal diagonal element. In the right-hand matrix the corresponding 


Spaces are left blank. This is simply another way of writing an upper 
triangular matrix, Where no values are written in below the principal 


are 0. 
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Lower triangular matrix 


As you might well guess, we can also have a triangular matrix in which 
all elements above the principal diagonal are zero. This is a lower triangular 
matriz, shown in Fig. 3.9.2. 


6 0. 0 6 

3 4 0 or 3 4 

5 2 6 o0 2-0 
Fic. 3.9.2 


For the lower triangular matrix also we may write in the zeros above the 
principal diagonal or omit them. In a lower triangular matrix an element 
is zero if its column number is greater than its row number. 

If we let letters stand for the numbers in a triangular matrix, we ordi- 
narily omit the zero entries, as in Fig. 3.9.3. 


а а» аз au 
а» Az ал ax 
33, Qa аз 33. 
Fig. 3.9.3 


The left-hand matrix is an upper triangular matrix and the right-hand one 
is a lower triangular matrix. A single letter may designate either an upper 
or lower triangular matrix. We may say simply, let a be an upper or lower 
triangular matrix, as the case may be; or, for an upper triangular matrix, 
We may say that a is a matrix such аба; is 0 if i is greater than j. Fora 
.OWer triangular matrix, we may say that a is a matrix such that a;; is 0 
if j is greater than i. In the behavioral sciences, triangular matrices are 
used in intermediate stages of the solutions of prediction problems. 


3.10 Partial Triangular Matrices 


Upper partial triangular matrix 


Another kind of matrix that is very common and useful in the behavioral 
Sciences is the partial triangular matrix. A partial triangular matrix is not 
4 Square matrix. As we have two types of triangular matrices, so we have 
two types of partial triangular matrices, namely the upper partial triangular 
Matrix and the lower partial triangular matrix. An upper partial triangular 
Matrix is shown in Fig. 3.10.1. 


2.9 T 9. ЕВ 
036552 
OO: 849 


Fie. 3.10.1 
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'This is а horizontal matrix in which an element is 0 if its row number is 
larger than its column number. In an upper partial triangular matrix all 
elements in the lower left hand corner are 0. 


Lower partial triangular matrix 


A lower partial triangular matrix is one in which an element is 0 if its 


column number is greater than its row number. Figure 3.10.2 is a lower 
partial triangular matrix. 


600 
3.40 
526 
28 9 
9 5 8 
Fig. 3.10.2 


A lower partial triangular matrix is always a vertical matrix. The 0 
elements are in the upper right hand corner of the matrix. Like the triangu- 
lar matrix, the partial triangular matrix typically arises in solutions to 


prediction or estimation problems that involve a number of predictor and 
criterion variables. 


3.11 The Zero-One Matrix 
The general case 


One general kind of matrix used extensively in the behavioral sciences 
is the zero-one matrix. Each element in this matrix is either 0 or 1 (Fig. 


3.11.1). 
0 x d 9 
0010 
tae b 
Оо 
1-737 Ое 
Fic. 3.11.1 


ement corresponding to that 
ers an item incorrectly, the element 
incorrectly answered item is 0. For 
matrix that you might get from sociological 
different families, Suppose the attributes or 


А If he answ 
corresponding to that person and the 
another example of a zero-one 


data, suppose the entities are 
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columns represent items indicative of the families’ standard of living, such 
as automobiles, telephones, electric refrigerators, electric dishwashers, 
television sets, and so on. If a family possessed one of these items, you 
would put a 1 in the appropriate column for that family. If it did not 
Possess the item you would put a 0. There are many other ways in which 
Zero-one matrices may arise from attributes describing or characterizing 
large numbers of people or entities. The zero-one type of matrix is ex- 
tremely useful not only because in it data can be represented simply, but 
also because operations with it are relatively simple. 


The permutation matrix 


Several kinds of zero-one matrices are not data matrices but are never- 
theless useful and important. One of these is the permutation matrix. It 
Serves a most important function in applications of matrix algebra to 
experimental data. It is a square matrix; each column in the matrix has a 
Single 1 and all the other elements in the column are 0. Similarly one of 
the elements in a row is 1 and all the rest of the elements in the row are 0. 
A permutation matrix is shown in Fig. 3.11.2. 


Pat S08 
0 BD 1-0 
Lo 5 
0 0 5 I 
Fig. 3.11.2 


You see that in this example each row has only one 1 in it, as does each 
Column. Notice that according to the definition, the identity matrix is a 
Special case of a permutation matrix. The identity matrix is one in which 
all nondiagonal elements are 0 and all diagonal elements are 1. This of 
Course means that each row and each column has a single 1 in it and all of 
the other elements are 0. Since the permutation matrix is so generally 
Useful in matrix algebra, we use the special symbol, т (Greek letter pi), to 
represent it. 

You recall that the order in which both entities and attributes are 
arranged may be more or less arbitrary. The permutation matrix may be 
Used to rearrange the order of either or both. In general it is used to 
rearrange rows or columns of any matrix. 


3.12 The ej Matrix 


A special case of a zero-one matrix is one all of whose elements are 0 
except one and that is simply the number 1. This rather peculiar matrix 
285 some practical uses in isolating and rearranging specific rows or columns 
1 a matrix. An e; matrix is given in Fig. 3.12.1. 
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0 0 0 0 
со rig 
0 0 ^0 "0 
0 оо 0 
ой 0 0 
Fig. 3.12.1 


Here we have a 5 by 4 matrix in which all elements are 0 except the element 
in the second row, third column. This element is 1. 

To designate a particular ег; matrix we use numbers for the subscripts t 
and j to indicate the row and column in which the unit element is found. 
For example, Fig. 3.12.1 is an e matrix. 


3.13 The Zero (Null) Matrix 


Just as in scalar algebra, or for that matter, ordinary arithmetic, zero is à 
number, so also in matrix algebra there is a zero matrix. As you would 
guess, a zero matrix is one in which every element is 0, as in Fig. 3.13.1. 


000 
000 
000 
0 à 0 
Fic. 3.13.1 


Here we have a 4 by 3 zero matrix. Another name more commonly used by 
mathematicians for a zero matrix is the null matrix. Since this term has 
come into such general usage, we shall also use it. 

F'rom the definition of a null matrix you see that it can be of any order. 
Ordinarily we indicate a null matrix by a simple 0 and specify its order. 
For exemple, we say let 0 be a 3 by 4 or n by m null matrix. 


3.14 Vectors 


A matrix may consist of only one row or one column. This form, called 
a vector, has many important uses in scientific prediction. The two general 


bens of vectors are of course the column vector, a matrix of only one 
column, and the row vector, a matrix of only one row. Figure 3.14.1 is 2 
column vector. 


9 
12 
13 


Fic. 3.14.1 
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Figure 3.14.2 is a row vector. 
{10 12 13 14] 
Fig. 3.14.2 


The order of a vector is specified simply by indicating the number of its 
elements. For example, Fig. 3.14.1 is a third-order column vector. Fig. 
3.14.2 is a fourth-order row vector. 

The vector is often the type of matrix in which we are most interested in 
problems of scientific prediction. For example, in Chapter 1, you recall, 
we talked about the weights to be applied to the predictor attributes in 
order to reconstruct or predict the criterion attribute. These weights may 
be represented as a vector in which the weight for each attribute is an 


element of the vector. 
When letters are to stand for the elements of a vector, we need only one 
subscript, since there is only one row or column. In Fig. 3.14.3 the elements 


of a row vector are represented by letters. 


[a a as] 
Fra. 3.14.3 


In Fig. 3.14.4, the elements of a column vector are letters. 


by 
bs 
bs 
bs 


Fig. 3.14.4 
This is a fourth-order column vector. Д 
We use the same method to indicate that a vector is of order n that we 


Used for other matrices. On the left in Fig. 3.14.5 is an nth order row 
Vector, and on the right, an nth order column vector. 


bı 
[ax de . аһ) by 
bn 

Fic. 3.14.5 


It is not necessary, however, to write in the second element of these vectors. 
To simplify the notation, we can write these vectors including only the first 
and last elements, as in Fig. 3.14.6. 
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bi 
HE axe. -0Д 9 
b, 
Fic. 3.14.6 


Since the vector has so many special properties, and since it is convenient 
to recognize it wherever it is used, we ordinarily use a distinctive symbol 
for it, namely V or v. To indicate different vectors, we can use different 


subscripts. For example, in Fig. 3.14.6 we can represent the right-hand 
vector by V». 


3.15 The Unit Vector 


We may have a vector in which all the elements are 1 or unity. This is 
called the unit vector. The unit vector is extremely useful in the analysis of 
experimental data to sum rows or columns of а matrix. It may be either a 
row or column vector. In Fig. 3.15.1, row and column unit vectors are 


given. 
1 
ЙИ ed e 1 
1 
Fig. 3.15.1 
The unit column vector is indicated simply by the number 1, as in Fig. 
3.15.2. 
1 
12|1 
1 
Fig. 3.15.2 


To show a unit vector is a TOW, we put a prime after the 1 as in Fig. 3.15.3. 
1S 00] 
Fic. 3.15.3 


You will see in Cha 
To indicate the 
example, we could 


pter 4 why we use the prime for this purpose. 
order of a unit vector, we may use a subscript. For 
use a subscript 3 to indicate a unit vector of order 3 as in 
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Ordinarily, however, the way in which the vector is used serves to indicate 
its order. 


3.16 The Sign Vector 


The sign vector is similar to a unit vector, but one or more of its elements 
15 minus 1, as shown in Fig. 3.16.1. 


Fig. 3.16.1 


Sign vectors may be either row or column vectors. They are useful in 
factor analysis, and in general, where some rows or columns are to be 


added and others subtracted. 


3.17 The Zero-One Vector 


Asa special case of a zero-one matrix, we may also have zero-one vectors. 
n a zero-one vector each element is either 0 or 1, as in Fig. 3.17.1. 


Fig. 3.17.1 


A variety of zero-one vector is a vector all of whose elements are 0 
except one, and that is unity. This we call an e; vector, or, sometimes, an 
elementary vector. In Fig. 3.17.2, row and column e; vectors are given. 

he €i vector is used extensively in the analysis of data from the behavioral 
Sciences, It serves to segregate rows, columns, and elements of a matrix. 


his is explained in Section 10.17.) 
[D 1 0 0] 0 


Fig. 3.17.2 


The subscript č in the symbol e; indicates that the ith element is unity. To 
Specify exactly the location of the unity element in the vector, we use a 
number for a subscript. Thus, in Fig. 3.17.2, we have on the left a fourth- 
Order e, vector and on the right a third-order ез column vector. 
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3.18 The Zero Vector 


A zero or null vector is actually а null matrix having only one row or 
column. Therefore all elements in the null vector are 0. Row and 
column null vectors are given in Fig. 3.18.1. 


0 
[D 0 0 90] 0 


Fig. 3.18.1 


We indicate the null vector simply by 0, defining the symbol as a vector 
to distinguish it from a matrix. The order of the null vector may be 
specified by a single subscript on the 0 or by a statement of the order. For 
example, we can say, let 0; be a third-order column vector. Ordinarily, 
however, the way in which the null vector is used will make clear its order. 
Then the order need not be specified. 


3.19 The Scalar Quantity 


A matrix with only one row and one column is called a scalar quantity, ОТ 
simply a scalar. So you see the letters you encountered in ordinary algebra 
are matrices of one row and one column, or scalar quantities. In the same 


way, the numbers you learned about in arithmetic are matrices of one row 
and one column, or scalars. 


To distinguish between a scalar quantity and a symbol for a general 
rectangular matrix, some texts print scalars in italics and the matrix 
symbol in bold face. In the case of hand written notations there seems to 
be no uniform practice. Ordinarily you can make the difference clear 
simply by defining certain letters as standing for scalar quantities, rather 
than matrices. This is the method used in this text. 


In matrix algebra in general and in this book, we shall use scalar quanti- 
ties very freely along with the matrices we employ. 


SUMMARY 


1. The general rectangular matrix: 
а. The height and width are different. 


b. The data matrix is usually rectangular and typically, though not 
necessarily, vertical. 


2. The square matrix: 
а. The square matrix is rarely a data matrix. 
b. In the behavioral sciences it is usually derived from a data matrix. 
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- Symmetric matrices: 


à. А symmetric matrix is a special case of a square matrix, in which 
corresponding elements of corresponding rows and columns are 
equal. 

b. In a symmetric matrix with elements a;;, we have Qj = Oy 

C. A symmetric matrix is sometimes written with only the principal 
diagonal and the elements either above or below it. This is in- 
correct and can lead to confusion with a triangular matrix. 


- The diagonal matrix: 


a. The diagonal matrix is a special case of a symmetric matrix in which 
all nondiagonal elements are zero. | - - 

b. It may be written without entries in the off. diagonal positions. 

с. The diagonal elements of a diagonal matrix may be written un- 
ambiguously with a single subscript to indicate their position. 

d. In this book the following symbols will usually be used to indicate 
diagonal matrices: D, d, A, б. 

The scalar matrix: ‘ а 

а. This is a special case of a diagonal matrix in which all the diagonal 


elements are equal. . ; 
b. The diagonal — may all be indicated with the same letter 


Without subscripts. 


The identity matrix: . : 
а. This is a special case of the scalar matrix in which the diagonal 


elements are unity. M 
b. In this book the identity matrix will be indicated by J, never by 1 


аз in some texts. — T" 
©. The identity matrix serves the same function in matrix algebra as 1 


in scalar algebra. -— 
d. The order : a diagonal matrix may be indicated verbally or by a 


subscript. 


* The sign matrix: 


а. This differs from the identity matrix only in that some of the 
diagonal elements may be — 1. 

b. It is indicated in the book by т. 

The skew symmetric matrix: e | 

а. This matrix has zeros in the main diagonal, and corresponding 
elements above and below the diagonal are equal in absolute 
magnitude and opposite in sign. 

- In a skew symmetric with elements a;,, we have а;; = —aj. 

Triangular matrices: ‘ | 

a. A square matrix with all 0 elements above the main diagonal is 
called a lower triangular matrix. 
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b. A square matrix with all zeros below the main diagonal is called an 
upper triangular matrix. 

Partial triangular matrices: | 

а. A vertical matrix with 0 for each element whose column number is 


greater than its row number is called a lower partial triangular 
matrix. 


b. A horizontal matrix with 0 for each element whose row number is 
greater than its column number is called an upper partial triangular 
matrix. 

Zero-one matrices: 

а. This is a matrix all of whose elements are either 0 or 1. | 

b. A permutation matrix is a special case of a zero-one matrix in which 
each row and column has a single 1 in it. It is therefore square. 

The e;; matrix has all 0 elements save one which is 1. 

The null matrix: 

а. This is a matrix all of whose elements are 0. 

b. It is indicated by 0 and the dimensionality must be specified. 


с. The null matrix serves the same function in matrix algebra as zero 
does in scalar algebra. 


Vectors: 
a. This is a special kind of matrix with only one row or column. 


b. The order of a vector is indicated by a single number which is the 


number of elements in it. However, it is also necessary to specify 
whether it is a row or column. 


A letter designation for a vector need carry only a single subscript 
to indicate its position in the vector. 


The unit vector: 

a. This vector has unity for all its elements, 

b. A column unit vector is indicated by 1 and a row by 1’. 

€. The order of a unit vector may be indicated verbally or by means of 


a subscript, or may be evident from the context in which the vector 
is used. 


The sign vector: Each clement is either +1 or — 1. 
"The zero-one vector: 


а. This is a special case of a zero-one matrix having only one row ОГ 
column. 

b. The e; vector is a special case of a 
single 1 and that in the ith position. 

The null vector: 

a. This is a vector all of whose elements are 0. 


b. It is indicated by 0 together with the specifications of its order, 
verbal or by subscript. 


zero-one vector having only & 
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19. Scalar quantity 
a. This is a matrix with only one row and column—the kind of 
number familiar to most persons. 
b. It may be distinguished from a symbol for a general rectangular 
matrix by a particular type style or, for a particular problem, by 
verbal definition of notation. 


EXERCISES 
Given the following matrices: 
L[2 3 4 1 2. F3 d 3. [4 ale d 
i 3 0: 2 p 3 ant Qin 
4 0 q5 006 7 
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4 759 @ ЫБ 1 g-2 4 33 
9 1 4 3 1 8 08 86 
9| 1 )10F15 0 oF 3ir[10 90] 12 1 0 9 
1 0-3. 0 240 
1 004 5 8 € 
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8 51 
13. f3 o o 14.f0 1.1. @ (oll арна 9.0 
озо 1 0040 бй 
0 03 Je OUT ^al 0б 1 
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ОО tO 
Mel a0] aspe] doef 0 0] 2:70 ao] seta 
001 0 zu 0 000 
100 0 Of o rd 000 
ie NE ue" 23.[0 0 
Tw dr cg 0 0 
4 x p 0 0 
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For each of the following terms, give the number of the matrix above that 
corresponds to it. Give one and only one number for each letter. 


(a) Rectangular (n) Sealar matrix 
(b) Lower triangular (о) т 

(c) Unit vector (p) Sealar 

(d) Square (д) I 

(e) Upper partial triangular (г) e 

(f) e; vector (s) + 

(g) Symmetric (t) Null matrix 

(h) Lower partial triangular (u) Skew symmetric 
() Sign vector (v) Upper triangular 
(k) D (w) Row vector 

(1) Zero-one matrix (x) Column vector 


(m) Null vector 
ANSWERS 


(а) 1, (b) 5, (е) 9, (4) 4, (е) 8, (0) 12, (в) 7, (b) 11, (j) 15, (k) 10, (1) 14 


(т) 18, (n)13, (о) 17, (р) 21, (q)16, (г) 20, (s) 19, (t) 23, (u) 22, (У) 2. 
(w) 6, (х) 3 


Chapter 4 


The Transpose of a Matrix 


Ss Se of the transpose of a matrix is usually dealt with in one 
oy or at most one paragraph in the majority of texts on matrix 
he н Аз а consequence of this cursory treatment, mathematicians not 
the x rud become confused when they attempt to use the transpose in 
isk x п of problems. Unless you thoroughly master the concept and 
бее ii for the analysis of experimental data, you may well make 
bonn s many man-hours or days of computational labor, or many 
reds of dollars in high-speed computor time. 
It is advisable first to consider two related concepts: the equality of 


ENA А 
atrices and the natural order of a matrix. 


4.1 Equality of Matrices 


Tas matrix algebra, as in scalar algebra, the concept of equality is highly 
ii Portant. Very simply, two matrices are equal to each other if, and only 
» each element of one is equal to the corresponding element of the other. 


Study Fig. 4.1.1. 


Gu G2 bu biz 
an а |= | ba bos 
аз 32 bai бз 
а b 
Fic. 4.1.1 


т В і 
s matrix on the left we call a, and that on the right we call b. Then to 
Y а = b means that element an is equal to element bu, a; is equal to 


Lie is equal to ba, and so on. 
ccording to this definition of equal matrices, two matrices cannot be 


e 

qual to each other unless thay are of the same order; Study Fig. 4.1.2. 
a. 28 4 2 9 
D, 3 14 
3 4 


Ес. 4.1.2 
51 
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Here we have two matrices; the first is a 3 by 2 matrix, the second a2 by 3 
matrix. The elements in the first column of the left-hand matrix аге, in 
order, the same as the elements in the first row of the right-hand matrix. 
Similarly, the elements in the second column of the left-hand matrix are, in 
order, the same as the elements in the second row of the right-hand matrix. 
Nevertheless, the two matrices cannot be said to be equal. They are not of 


the same order, nor is the ijth element of one in general equal to the ¿jth 
element of the other. 


4.2 The Natural Order of a Matrix 


Although mathematicians do not talk about the natural order of a matrix, 


the idea is very useful in applying matrix algebra to large masses of data in 
the social sciences and elsewhere. 


Natural order of data matrices 


For data matrices, we shall say that the elements are in their natural 
order when the entities are rows and the attributes are columns. Usually, 


therefore, the natural order of a data matrix has more rows than columns; 
it is usually a vertical matrix. 


Natural order of other rectangular matrices 


In dealing with a rectangular matrix for which the entities and attributes 


are not specified, we usually take the natural order to be the vertical form 


of the matrix. Since a vector is a special case of a rectangular matrix, we 


can regard the natural order of a vector as the column form. In this book, 
unless we specifically state otherwise, the column vector is regarded as the 
natural order of the vector. 

The partial triangular matrix, described in Section 3.10 is a rectangular 
matrix, We may say that the natural order of a partial triangular matrix 
is the lower partial triangular matrix. You recall that a lower partial 


triangular matrix is vertical. Thus the natural order of a partial triangular 
matrix is shown in Fig. 4.2.1, 


w со — Oo MD 
-ї л т ҥ о 
нњ оо о 


Fie. 4.2.1 
Natural order of square matrices 


There is only one kind of Square matrix 


| for which we specify a natural 
order. This is the square triangular matrix d 


. We call the lower triangular 
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matrix the natural order of the triangular matrix. The reason for this you 
have probably guessed. It is because the triangular matrix is a special case 
of the partial triangular matrix. Since the lower partial triangular matrix 
Is regarded as the natural order, we shall, for the sake of uniformity, regard 
its special case, the lower square triangular matrix, also as the natural 
order. In Fig. 4.2.2 are shown two examples of the natural order of a 
triangular matrix. 


170720 ац 

2 9. D an ax 

B. 1-1 аз 03 Ass 
Fic. 4.2.2 


Үоц see, therefore, that in the natural order of а triangular matrix the 
first Subscript is never smaller than the second. 

In Speaking of the square matrix we usually have no way of specifying 
the natural order. Very rarely is a data matrix a square matrix. Further- 
More, in operations on data matrices or other derived matrices, we have 
very little use for square matrices in general. Several special types of 
Square matrices are used; these will be discussed in later chapters. One, 
however, with which you are already familiar (Section 3.11) is the permuta- 
Чоп Matrix, a zero-one matrix with a 1 in each row and column, all the 
Temaining elements being 0. We do not define the natural order of а 
Permutation matrix. 

Other types of square matrices with which you are already familiar are 
the Beneral symmetric matrix and various special cases of it—the diagonal 
matrix, the scalar matrix, the identity matrix, and the sign matrix. We 
do not define the natural order of any form of symmetric matrix, since 
Writing rows for columns or vice versa would not change the value of a 
Symmetric matrix. 


43 What the Transpose of a Matrix Is 


By this time you may have guessed what we mean by the transpose of a 
Matrix. One matrix is called the transpose of another if the rows of the first 
are written as the columns of the second. In Fig. 4.1.2, the second or 
right-hand matrix is the transpose of the first. You will see that the rows of 
the first form the corresponding columns of the second. Another example 
of a matrix and its transpose is given in Fig. 4.3.1. 


3 2 2529 T 
2 1 2 1 8 
37 9 

Fia. 4.3.1 
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ix is the transpose of the first. Of course, we can also 
biis pim T iovis a the second. Notice that if the rows of 
ks pe are the columns of another, then also the columns of the first 
npe must be the rows of the other. These are two different ways of 

i e thing. 
Ec S of defining the transpose of a matrix is to say that the 
а} element of the matrix is the a;;th element of the transpose. ҮШ. 
example, in Fig. 4.3.1, suppose that т is 3 and j is 1. Then, а мош а 
азі, which in the left-hand matrix is 7; aj; would be a, which in the right- 
hand matrix is also 7. УЖ 

It should be clear then that the order of the transpose of а matrix is the 
reverse of the order of the original matrix. If a matrix is of order 3 by 4, 


then its transpose is of order 4 by 3; or, in general, if the order of a matrix is 
п by m, then the order of the transpose is m by т. 


4.4 The Natural Order and the Transpose 


Since the natural order of a matrix is usually vertical, the transpose of 
its natural order is usually horizontal. Also, since in the natural order of à 
data matrix the entities are usually rows and the columns attributes, we 
think of the transpose of а data matrix as ordinarily having rows for 
attributes and columns for entities. In general, therefore, we think of the 
transpose of a data matrix as being usually horizontal. 


4.5 The Symbol for the Transpose 
When a letter stands for a matrix 


When we let a single letter stand for a matrix, it is customary to put 9 
prime after it to refer to the transpose of the matrix. Е 

how we ordinarily let the un 
the primed letters for its tr 


igure 4.5.1 illustrates 
primed letters stand for the original matrix an 


anspose. 
4 $8 EMEN 
a=|2 1 =F 180 le xO) 
т 9 
Fig. 4.5.1 


You see now why in Chapter 


i 3, Section 15, we let the number 1 stand for 2 
column unit vector and 1' 


Stand for the unit row vector. 


Letters with double subscripts for elements 


If we let letters with double Subseripts stand for the elements of the 
matrix, we sometimes reverse our subscript procedure to indicate the 
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transpose of а matrix. For example, if a is given as on the left of Fig. 4.5.2, 
then its transpose a’ would be given as on the right. 


ап Ae atis ап ал Qn 
а = | ал аз Qm аз аз 
аз Q3 
Fig. 4.5.2 
Qu Gig Qim ап Qa Qni 
@ = а Ax ... Am UPS Qm Qd» ... аһ 
Ani An ... Qnm Qim Am ... Qnm 
Fic. 4.5.3 


You see that on the right of Fig. 4.5.2 the first subscript refers to the 
Column in which the element is found and the second refers to the row. 
This is another of the exceptions to the rule given in Chapter 2, by which 
the first figure in the subscript represents the row and the second figure 
represents the column. 

In general, if we let a stand for the left-hand matrix in Fig. 4.5.3, then 
а, its transpose, would stand for the right-hand matrix. Notice particularly 
їп Fig. 4.5.3 that the lower right-hand element in the matrix a is the same 
as the lower right-hand element in a’. This element in both cases lS алт. 
However, in the first matrix n refers to the row and m to the column, 
Whereas in the second matrix n refers to the column and m to the row. 


When subscripts are used to indicate order 


е You may recall that in Chapter 2 we said the order of а matrix is some- 
times indicated by double subscripts to a letter that stands for the entire 
matrix. If a matrix has n rows and m columns, we may simply let anm 
represent this entire matrix. If we do use this double subscript notation, 
the transpose of the matrix is one with m rows and n columns. We would 
Indicate this relationship as shown in Fig. 4.5.4. 


, 
Anm = Amn 


Fic. 4.5.4 


Here you see that putting a prime after a letter has the effect of inter- 
changing the subscripts of the matrix. We could translate Fig. 4.5.4 into 
Words by saying that the transpose of a matrix with n rows and m columns 
18 a matrix with m rows and n columns. 
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0 0 Oo} ооо 

ооо = ооо 

„00 n o uU 
Fic. 4.7.5 


The scalar quantity (Section 3.19) is a symmetric matrix with only one 
row and column—that is, а single number. Of course, if the row were 
written as a column, or the column as a row, this matrix would not be 
altered. "Therefore, the transpose of a scalar quantity is the same scalar 
quantity. This is so obvious that you may wonder why it should be stated 
во carefully. Sometimes, however, this simple and obvious rule is very 
useful in simplifying complex solutions or proofs. We can express this rule 
by 

аз = ai (4.7.5) 
where а;; 


simply means the element in the ith row and jth column of à 
matrix, n 


о matter what the size of the matrix. To use a simple ordinary 
number instead of the letter, we could write the same rule as 


Fag (4.7.6) 


This equation merely says that the transpose of the number 3 is the num- 


ber 3. 


4.8 The Transpose of a Vector 


You will not have to learn anythin 
vector. A vector, as you know, is si 


means a row and the primed symbol a column 
vector, 


It must also now be clear t 
vector. Figure 4.8.1 illustra; 
letter to indicate a column а 


hat the transpose of a row vector is a column 
tes the use of the 


pectively. 
2 
=. Vaj 4 3] 
3 


Fic. 4.8.1 
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We indicate the transpose of special kinds of vectors in the same way. 


For example, Fig. 4.8.2 illustrates an е: vector (Section 2.17) and its 
transpose. 


é-[l 0 0 0] 


ооо н 


е; 


Fic. 4.8.2 


Figure 4.8.3 illustrates а sign vector (Section 2.15) and its transpose. 


1 

—1 
m /-—[l —1 —1 I 
i= i =f ] 


1 
Fig. 4.8.3 


The null vector (Section 2.18) and its transpose are illustrated in Fig. 4.8.4. 
б =|6 0’=(0 0 0] 


Fig. 4.8.4 


SUMMARY 


1. Equality of matrices: 3 : Я 
2. Two matrices аге said to be equal if and only if all corresponding 


elements are equal. З 
b. Therefore matrices cannot be equal unless their orders are the same. 


2. Natura] order of a matrix: ER 
2. Data matrices. In this book the natural order of a data matrix is said 


to be such that rows are entities and columns attributes, 

b. Other rectangular matrices. Matrices that are not data matrices are 
Usually considered in natural order if they are vertical. The natural 
Order of a vector, unless otherwise specified, is the column form, 

©. Square matrices. The only square matrix to be assigned a natural 
Order is the triangular matrix. Its natural order is taken as the lower 
triangular form. 

3. The transpose of a matrix: 

a. The transpose of a matrix is one in which the rows are written as 
Columns or the columns as rows: therefore the ijth element of the 
matrix becomes the jith element of the transpose. 
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b. The order of the transpose of a matrix is therefore the reverse of the 
order of the matrix. 
4. The natural order and the transpose. | 
a. The transpose of the natural-order data matrix usually has attributes 
for rows and entities for columns. A 
b. The transpose of a natural-order rectangular matrix is usually а 
horizontal matrix. 
5. The symbol for the transpose: 


a. If a letter stands for a matrix it is primed to indicate its transpose: 
a’ is the transpose of a. 


b. If the natural order is defined, the letter representing it is unprimed 
to indicate it is the natural order. 


c. The double subscript notation in which the first subscript stands for 
column and the second for row may be reversed to indicate the 
transpose. 

6. The transpose of a transpose: 


a. If the transpose of a matrix is transposed the original matrix is 
restored. 


b. This is indicated by [a’]’ = a. 


7. Transposes of common kinds of matrices: 


a. The transpose of a vertical matrix is horizontal and vice versa. 
b. The transpose of a square matrix is a square matrix. 
c. The transpose of a lower triangular matrix is an upper triangular 


matrix and vice versa, The same rule holds for partial triangular 
matrices. 


- The transposes of all symmetric matrices are the same as the original 
matrices. This includes: general symmetric, diagonal, scalar, identity, 
and sign matrices. Obviously it also includes scalar quantities. 

8. The transpose of a vector: 


а. The column vector is written without the prime. 
b. Its transpose is a row vector. 


c. The primed symbol for a vector means a row vector unless otherwise 
specified. 
EXERCISES 


1. Write the transposes of each of the matrices given in the exercises for Chapter 3- 
2. Give the numbers of those whose transposes are vertical. 


3. Give the numbers of those whose transposes are different but whose orders are 
unchanged. 


4. Give the numbers of those whose transposes are horizontal. 


5. Give the numbers of those whose transposes leave the matrix unchanged. 


THE TRANSPOSE OF A MATRIX 


ANSWERS 


- (1), (6), (8), (12), (15) 

- (2), (4), (8), (17), (20), (22) 

< (3), (9), (11), (14), (18), (23) 
- (7), (10), (13), (16), (19), (21) 
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Chapter 5 


The Supermatríx 


We have seen that the operation called transposition, though not needed 
in scalar algebra, is important in matrix algebra. Another operation used 
only in matrix algebra gives rise to what are called supermatrices. This 
operation, like transposition, is simple and frequently is given only sum- 
mary treatment by the mathematicians. It is, however, essential in the 
construction of experimental designs in the behavioral sciences, and its 
competent utilization in such research can be very helpful. Therefore we 
shall develop in detail the concept of the supermatrix. 


5.1 Definition of a Supermatrix 


The supermatrix and its elements 


Before trying to define а supermatrix, let us designate the kind of 
matrices we have been talking about up to now as simple matrices. By & 
simple matrix we mean a matrix each of whose elements is just an ordinary 
number or a letter that stands for the number. In other words, the elements 
of a simple matrix are scalars, or scalar quantities. 

A supermatrix, on the other hand, is one whose elements are themselves 
matrices with elements that can be either scalars or other matrices. In 
general, in the kind of supermatrices we shall deal with in this book the 


matric elements have only scalars for their elements, Suppose we define 
four matrices as in Fig. 5.1.1. 


Ея sa [oral 
n 3 7 Qi = 3 2 
8 1 
ва = [$ 4 o= |$ 3 
Fie. 5.1.1 


You will notice that we are using number subscripts on each of the letters 
that stand for entire matrices. This is one reason why we discouraged the 
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use of number subscripts to indicate order when a single letter was used to 
stand for a matrix (Section 2.5). You see now we want to use number 
Subscripts to stand for separate matrices. The reason we have chosen the 
particular number subscripts for each matrix in Fig. 5.1.1 will become clear 
if we let a stand for an entire matrix whose elements are themselves these 
four matrices, as in Fig. 5.1.2. 


ап de 
a= 
ал а» 


Fig. 5.1.2 


Note that we are using the same rule of double subscripts that we used for 
Simple matrices. The first subscript indicates the row in which the matric 
element is found and the second subscript indicates the column. 

We can write out the matrix a in Fig. 5.1.2 in terms of the original 
matrix elements in Fig. 5.1.1. This we do in Fig. 5.1.3. 


w to 
ENS 


6 
7 
Fic. 5.1.8 


Here the elements are divided vertically and horizontally by thin lines. 
Tf the lines were not used, the figure would be read as a simple matrix. ~ 
Thus far we have referred to the elements in a supermatrix as matric 
elements, It is perhaps more usual to call the elements in а supermatrix 
Submatricos. We speak of the submatrices within a supermatrix. 


Order and the supermatrix 


The order of a supermatrix is defined in the same way as that of a simple 
matrix, The height of a supermatrix is the number of rows of submatrices 
i it. The width of a supermatrix is the number of columns of submatrices 

it. 

As you may have guessed, all submatrices within a given row must have 
the same number of rows; likewise all submatrices within a given column 
Must have the same number of columns. A diagrammatic illustration of this 
Tule is pi, 5.1.4. 
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®ж шаб шаш 


EHE SH 
Fa 


Fie. 5.1.4 


E 


In the first row of rectangles we have one row of squares for each rectangle; 
in the second row of rectangles we have three rows of squares for each 
rectangle; and in the third row of rectangles we have two rows of squares 
for each rectangle. Similarly for the first column of rectangles we have two 
columns of squares for each rectangle. For the second column of rectangles 
we have three columns of squares for each rectangle, and for the third 
column of rectangles, we have four columns of squares for each rectangle. 

One thing should now be clear from the definition of a supermatrix. The 
superorder of a supermatrix tells us nothing about the simple order of the 
matrix from which it was obtained by partitioning. Furthermore, the order 


of a supermatrix tells us nothing about the orders of the submatrices 
within that supermatrix. 


5.2 Partitioned Matrices 


It is always possible to construct a supermatrix from any simple matrix 
that is not a scalar quantity. 


The supermatrix from the simple matrix 


The process of constructing a supermatrix from a simple matrix is called 
partitioning. A simple matrix may be partitioned by dividing or separating 
the matrix between certain specified columns, and then between certain 
specified rows. Or the procedure may be reversed. The division may be 


made first between rows and then between columns. To illustrate, suppose 
we have a simple matrix as in Fig. 5.2.1. 


3 


а= 


4 
3 
2 
7 


№ ел Lb 
Ny PN 


6 
1 
4 


Fig. 5.2.1 
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This is a 4 by 4 matrix with numbers for elements. Now let us draw а 
thin line between the first and second columns. This gives us the matrix 


a pago 
ES EE 
орка 52 

rlw 

Fig. 5.2.2 


Actually now the figure may be regarded as a supermatrix with two matric 
elements forming one row and two columns. However, we shall next 
divide the matrix between rows 2 and 3 to give us Fig. 5.2.3. 


Fic. 5.2.3 


It is now a second-order supermatrix, with two rows and two columns of 
submatrices. If we define the submatrices as in Fig. 5.2.4, we can write 
them simply as in Fig. 5.2.5. 


\ 
‚р. 
о њ 
LJ 
e 
5 
Ш 
D—— 
ow 
- t 
њ ~ 
LA 


(227 = 
2 1 5 2] 
an = | 7 nc pup wm 
Fia. 5.2.4 
E. fes E 
x аз Ax 
Fic. 5.2.5 


The elements now are the submatrices defined in Fig. 5.2.4, and therefore, 
Fig. 5.2.5 is, in terms of letters, the supermatrix given in Fig. 52:8. ; 
According to the methods we have just illustrated, a simple matrix can 
© partitioned into a supermatrix in any way that happens to suit our 
Purposes, 


$3 The Natural Order and the Supermatrix 


You will recall that usually in working with simple matrices we take the 
Dàtural order to be the vertical form of the matrix. In working with the 
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supermatrix, we have no simple rule for determining its natural order. 
'The natural order of a supermatrix is usually determined by the natural 
order of the corresponding simple matrix. Furthermore, we are not usually 
concerned with the natural order of the submatrices within a supermatrix. 
Here again, if it becomes necessary or convenient to define the natural 
orders of these submatrices, we take their natural orders to be their orders 


in the natural order of the simple matrix from which they were constructed 
by partitioning. 


5.4 Symmetric Partitioning 


By the symmetric partitioning of a matrix we mean that the rows and 
columns are partitioned in exactly the same way. If the matrix is parti- 
tioned between the first and second columns and between the third and 
fourth columns, then to be symmetrically partitioned it must also be 
partitioned between the first and second rows and the third and fourth rows. 

According to this rule of symmetric partitioning only square simple 


matrices can be symmetrically partitioned. An example of a symmetrically 
partitioned matrix is given in Fig. 5.4.1. 


Fig. 5.4.1 


Here you see that the matrix has been partitioned between columns one 
and two, three and four, and four and five. 


between rows one and two, three and four, an 

One fact you should notice in Fig. 5.4.1 (if 
this conclusion yourself) is that some of the e 
be scalar quantities. Here the first element i 
is 4, ora single number. This is also true of th 
the third and fourth columns. You can also 
Fig. 5.4.1 that are scalars. Notice also that some of the submatrices аге 
row vectors and some are column vectors, 

While it is possible to partition symmetrically any square simple matrix, 
this type of partitioning is usually performed on simple matrices that ате 
themselves symmetric. That is, there is nothing in matrix algebra itself (0 
cause us to prefer the symmetric partitioning of symmetric simple matr! ices 


It has also been partitioned 
d four and five. 

you have not already reached 
lements in a supermatrix may 
n the first row of submatrices 
e elements in the first row an 
find other matric elements 11 
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rather than general square matrices, but it usually happens that in the 
analysis of data, the matrices we can usefully partition symmetrically are 
symmetric simple matrices. 


5.5 The Symmetric Partitioning of a Symmetric Simple Matrix 


The general case 


Let us take a fourth-order symmetric matrix and partition it between the 


Second and third rows and also the second and third columns, as in Fig. 
5.5.1. 


Ев. 5.5.1 


We can represent this matrix as a supermatrix with letter elements. The 
Submatrices are then as shown in Fig. 5.5.2. 


4 3 Ju 
а = 3 6 а= i 4 
97] 5 2 
ay = 7 4 а» = 27 


Fia. 5.5.2 


We can therefore write the matrix a with letter elements representing the 
Submatrices, This we do in Fig. 5.5.3. 


za, | dt e] 
Lim ал @% 
Fia. 5.5.3 


Notice that the diagonal elements in Fig. 5.5.8 are ап and а». If you look 
at Fig. 5.5.2, you will see that an is a symmetric matrix and as is also a 
Symmetrie matrix. 
Furthermore the nondiagonal elements in Fig. 5.5.3 are the matrices 
x and ay. Referring again to Fig. 5.5.2, you see that a» is the transpose 
Ap, 
We therefore have two very simple rules about the matric elements of a 
Зу: mmetrieg]ly partitioned symmetric simple matrix. The first of these is 
at the diagonal submatrices of the supermatrix are all symmetric ma- 
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trices. The second is that the matric elements below the diagonal are the 
transposes of the corresponding elements above the diagonal. Figure 5.5.4 
is an example of a third-order supermatrix obtained from a symmetric 
partitioning of a symmetric simple matrix. 


Qn ar аз 

@=| Gis de Gs 
] 

Qj o» 33 


Fi. 5.5.4 


This figure illustrates the second rule. The elements below the diagonal 
have subscripts exactly the same and in the same order as their corre- 
sponding elements above the diagonal. However, to indicate the transposes, 
the elements below the diagonal have primes after them while those above 
do not. For example, the element in the first row and second column is аз, 
whereas the element in the second row and first column is aj» We can write 


the general expression for a symmetrically partitioned symmetric matrix 
as in Fig. 5.5.5. 


а Ay аһ 
+ 
a=|% 0 Aan 
+ ; ; i 
Qin Фи ..„ бм» 
Fic. 5.5.5 


If we were writing either Fig. 5.5.4 or Fig. 5.5.5, however, the reader could 
not tell that they are symmetrically partitioned symmetric simple matrices 
unless we also specified that ап, а», and so on, are symmetric submatrices: 
If in Fig. 5.5.5, for example, we said let а equal a;; then the reader woul 
know that a symmetric matrix had been symmetrically partitioned. 


The diagonal simple matrix 


If we want to indicate a sy 


3 mmetrically partitioned simple diagonal 
matrix, we could w. 


rite this as in Fig. 5.5.6. 


D 0 0 
Da OF “ps 0 
0 0 D, 

Fig. 5.5.6 


Here we would have to specify the order of the diagonal submatrices 
Di, D: to Р». The prime on each 0 below the diagonal means that its orde" 
is reversed from the order of the corresponding null matrix above 06 
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diagonal. Figure 5.5.6 is a precise way of indicating a symmetrically 
partitioned diagonal matrix or superdiagonal matrix. However, we ordi- 
narily need not go to the trouble of writing in the zeros. There will usually 
be no misunderstanding if we write the matrix as in Fig. 5.5.7. 


Di 
D: 


Dn 
Fig. 5.5.7 


The difference between Figs. 5.5.6 and 5.5.7 is that in the latter we have 
omitted the zeros or null submatrices. 


The identity matrix 


The procedure for partitioning an identity matrix is shown in Fig. 5.5.8. 


alae | 


Fic. 5.5.8 


The subscripts s and t indicate the number of rows and columns in the first 
ànd second identity matrices respectively. It is often convenient to use 
these subscripts to indicate the order of the identity submatrices. 
Although it is possible to perform nonsymmetrie partitioning of sym- 
metric simple matrices, the occasions when this is useful or convenient in 
е analysis of data are very rare. In general, symmetric simple matrices 
are subjected only to symmetric partitioning. 


5.6 General Diagonal Supermatrices 


‚ You have seen in the preceding section how we may partition а simple 
‘agonal matrix to get a superdiagonal matrix. In this case the matric 
‘agonal elements are themselves simple diagonal matrices. In certain 
experimental designs we may require superdiagonal matrices whose diagonal 
° ements may be any type of simple matrix. They may be square, hori- 
zontal, vertical, triangular, or symmetric. They may be row or column 
vectors of either general or special types, such as unit, ei, Or sign vectors. 
at the elements are depends on the particular experimental design and 
© mathematical model chosen for the analysis of the data. * 
«97 applications of a number of different superdiagonal matrices see Horst, Paul, 
neralized Canonical Correlations and Their Applications to Experimental Data,” 
Clin, Psychol., Monog. Supp. No. 14 (October 1961). 
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Figure 5.6.1 is an example of à general superdiagonal matrix whose matric 
diagonal elements are rectangular matrices. 


Fic. 5.6.1 


It may be written symbolically as in Fig. 5.6.2. 


im 0 
Bee 0) 
Fic. 5.6.2 


Notice that in the partitioning, the simple orders of the matric diagonal 
elements determine the simple orders of the null matric elements. (The 
subscripts here do not indicate order.) 

An example of а superdiagonal matrix with vector elements, which can be 
useful in certain experimental designs, is given in Fig. 5.6.3. 


1|0]0 
2492176 
ее; 
01110 
Di = |0 1 0 
Or) EO 
SUE 
010 [1 
Ой T^f. 
Fig. 5.6.3 


Here the diagonal elements are simply column unit vectors. 


Figure 563 
may be written symbolically as in Fig. 5.6.4. 


h. 0 0 
D-|0 1, 0 
ONG. 1; 

Fic. 5.6.4 


The subscripts on the diagonal unit vectors in F 


ur 
Dena ; ig. 5.6.4 indicate the? 
positions in the matrix D, and not the simple orde 


rs of the vectors. 
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5.7 The Partial Triangular Matrix as a Supermatrix 


А You are already familiar with the partial triangular matrix. It is some- 
times convenient to regard it as a supermatrix. This we do by partitioning 
it between the triangular part of the matrix and the rest of the matrix. 
In this way we have two submatrices, one of which is a square triangular 
mat rix and the other a rectangular matrix. If we partition an upper partial 
triangular matrix into two submatrices in this manner, we have a 1 by 2 
Supermatrix. The first clement in the row is an upper triangular matrix 
and the second element is a rectangular matrix. Ап upper partial triangular 
matrix partitioned in this fashion is shown in Fig. 5.7.1. 


Sw TI 48: 

01 2:11:29: 5 

0 0 5|4 9 
Fie. 5.7.1 


If we partition a lower partial triangular matrix in the manner just 
described, we have a 2 by 1 supermatrix. The first element in this super- 
matrix is а square lower triangular matrix and the second element is à 
rectangular matrix. A lower partial triangular matrix partitioned in this 
Way is given in Fig. 5.7.2. 


2. 0 0 
9 4 0 
8 3 6 
5 2 9 
4738 
Fra. 5.7.2 


ane convenient at times to use letters to indicate the submatrices of a 
rtial triangular matrix. In Fig. 5.7.3 we represent Fig. 5.7.1 in this way. 


B' = [T" | a] 
Frc. 5.7.3 


pre е have agreed that the natural order shall be that of the lower 
it is s triangular matrix, we use à prime on the B in Fig. 5.7.3 to show that 
of the e transpose of the lower partial triangular matrix. On the right side 
^ As ш we use Т” for the square triangular submatrix. The prime 
Tectan at it is an upper triangular matrix. We use an a to indicate the 
: Pes ai submatrix. We do not use a prime after the letter a because 
«іо аррепѕ that in Fig. 5.7.1 this submatrix has more rows than 

mns. Even though we have no hard and fast rules for indicating the 
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1 order in a supermatrix, it is often convenient to prime horizontal 
ea lements, particularly in the case of partitioned partial triangular 
idee "The separate submatrices of Fig. 5.7.1 that have been used in 
the right side of Fig. 5.7.3 are shown in Fig. 5.7.4. 


3.4 7 6 8 
T'—«|0 2 1 a=|2 5 
0 0 5 4 9 

Ес. 5.7.4 


In the same way, we can indicate with letters the submatrices in Fig. 5.7.2 
as in Fig. 5.7.5. 


200 ‚_ 53 9 
Т=|9 4 0 а' = 473 
8 3 6 
Fia. 5.7.5 


" Е іп 
Using the symbols as defined here, we can write the same matrix as 
Fig. 5.7.6. 


T 
c=|- 
a’ 
Fig. 5.7.6 


Here you see that for the C on the left we do not use a prime because pA 
regard the lower partial triangular matrix as the natural order. On i ie 
right side, the first submatrix is a lower triangular matrix which we Шш 5 
without a prime. The lower submatrix, a’, has a prime because in Fig. 5-1- 
we see that this matrix is horizontal rather than vertical. 
It would be quite possible in F 
only 2. In this case it w. 
in Fig. 5.7.3 to show th. 
it would be quite possi 


ig. 5.7.1 to have, say, 7 columns instead of 
ould be perfectly appropriate to use a’ instead 0 2 
at the matrix is horizontal. Similarly in Fig. S 
ble for the matrix below to have more rows tha 


: : one ix 
columns, in which case we could use, in Fig. 5.7.6, a for the lower submair 
instead of a’, to show that it was vertical. 


In most analyses of prediction 
matrix plays an important role. 
submatrix is the same as the numb 
tion matrix while the number of 


* А ar 
data matrices, the partial triangul Ё 
Usually, the order of the ipe 
ег of predictor attributes in the pre 
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lower; and therefore the standard form of the rectangular part is usually 
as given in Fig. 5.7.1 or Fig. 5.7.3. Therefore the number of rows in the 
rectangular submatrix is usually equal to the number of predictor attributes 
While the number of columns is usually equal to the number of criterion 
attributes, 


5.8 Supervectors 


Type | column supervectors 


Since a vector is a special case of a matrix with only one row or one 
column, we shall define a simple vector as we did a simple matrix. A simple 
vector is a vector each of whose elements is a scalar. It is useful, however, to 
define a number of different types of supervectors. The first of these types 
results from the partitioning of a simple vector. This we call a type I super- 
vector. The elements of a type I column supervector would themselves 
be column vectors, For example, Fig. 5.8.1 is a type I column supervector 
that has two column vector elements, each of which in turn has two scalar 
elements, 


1 
3 
VS wee 
4 
6 
Fic. 5.8.1 


The terms we use are analogous to those used in working with matrices; 
We call the vector elements of a supervector subvectors. Thus, in Fig. 5.8.1 
the first subvector of the vector V consists of the column elements 1 and 3. 

е second subvector consists of the column elements 4 and 6. We may 
also write a column vector in terms of its column subvectors as in 


Vi 
y= Vs 
y. 
Fra. 5.8.2 


In this figure Vi, V2, and V, are all column subvectors of the column 
Vector y. Each V; column subvector may have as many elements as we 
Please, depending on the particular problem we are working with. In 
Particular, one or more of these column subvectors may have only one 
element, 
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Type | row supervectors 


We may also have a type I row supervector, each of whose elements is 
a row vector, as in Fig. 5.8.3. 


y'-[ 314 6] 
Fig. 5.8.3 


In this figure, the first row subvector has the elements 1 and 3 and the 
second row subvector has the elements 4 and 6. You will see that on the 
left V has the prime to show that it is a row vector, since we consider the 
natural order to be the column vector. We can also use letters to stand for 
the row subvector elements of a type I row supervector as in Fig. 5.8.4. 


Wes "WE lle Val 
Fic. 5.8.4 


Here we have a type I row supervector with n row subvectors. You will 


notice that we use primes after each of the subvectors to show that they too 
are row vectors. 


5.9 Matrices Expressed as Supervectors 


We have discussed the type I supervector, which can be obtained by the 
partitioning of simple vectors. This partitioning, if applied to column 
vectors, yields column subvectors; and if applied to row vectors, yields 
row subvectors. We may also consider column supervectors whose elements 
аге row vectors, and row supervectors whose elements are column vectors. 
It is easy to see that vectors of this type when expressed in simple form are 
matrices. Such vectors we shall call type II supervectors. Let us see in 


greater detail how we may express a matrix as а type II supervector. 
We start with Eq. (5.9.1). 


аа ар Aim 
g= аа Ax Am (5.9.1) 
Oni баз ... Qim 


Type Il column supervectors 


Equation (5.9.1) is the general expression for an n by m matrix in 
expanded form. Now suppose we consider each row of the matrix in Eq. 


(5.9.1) as а separate row vector. Suppose we let a different symbol repre- 
sent each one of these row vectors as shown in 
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а = [ац ae ... аһ) 
= [ü йж Ll “hl (5.9.2) 
E 02.9.2 
d, = [аа аг ... аһ) 


We сап now substitute the left side of the equations in (5.9.2) into the 
right side of Eq. (5.9.1) to get 
ГА 
а. 
a 
a= > (5.9.8) 
М 
ап. Дь 
You see now that the right side of Eq. (5.9.3) is а column vector each of 
whose elements is a row vector. Equation (5.9.3) then is one way of writing 


a matrix as a column supervector whose elements are rows. 

Now let us discuss the meaning of the symbols used for the elements in 
the right side of Eq. (5.9.3). First, each element is primed to show that 
it is а row vector. Second, the first subscript in each element indicates the 
row in the original matrix from which the row vector is taken. Third, а 
dot is used in the second subscript position of each element to show that 
the element has a number of columns in it. Fourth, the subscript n for the 
last element means that the matrix has n rows. Fifth, the subscript m 
outside the brackets, or the external subscript as it is called, indicates the 
number of columns in the matrix, or the number of elements in each of the 
row vectors within the brackets. 

Later on in this book we shall frequently refer to a row of elements in a 
matrix as a row vector of the matrix. We shall speak of the first, the second, 
or the ith row vector of a matrix. In the same way we shall also speak of a 
column of elements in a matrix as a column vector of the matrix. 


Type Il row supervectors 


We shall next see that a matrix may also be expressed as а row super- 
vector with elements that are column vectors. First we go back to Eq. 
(5.9.1) and consider each of its column vectors. We let each of these 


column vectors be indicated by the symbols given in 


au аш Aim 
ат а» а» 

а = аз = Bigg, =| o (5.9.4) 
Ant ал? Anm, 


Now if we substitute the symbols on the left side of Eq. (5.9.4) into the 
matrix on the right side of Eq. (5.9.1), we have 


acr de 2... Gals (5.9.5) 
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Equation (5.9.5) expresses the matrix a as a row supervector whose sub- 
vectors are the column vectors of the original matrix given in Eq. (5.9.1). 

Be sure now that you understand clearly the meaning of the symbols on 
the right side of Eq. (5.9.5). First, each of the elements in the row is left 
unprimed. The fact that primes are not used after these elements shows 
that they are column vectors. Second, the first subscript instead of the 
second is a dot. It is used to show that the row elements, rather than the 
column elements, vary. Third, the second subscript after each element 
indicates the column of the matrix from which the vector is taken. Fourth, 
the second subscript of the last element is m and indicates the number of 
columns in the matrix. The subscript outside the brackets, n, or the ex- 
ternal subscript, shows the number of rows in the matrix. 

Notice then that we may equally well write a matrix as a column super- 
vector whose elements are rows, as we did in Eq. (5.9.3), or as a row 
supervector whose elements are columns, as we did in Eq. (5.9.5). There- 
fore from Eqs. (5.9.3) and (5.9.5), we can write 

а. 
ГА А 
Е E UES cans Gale (5.9.6) 
РИА 
Equation (5.9.6), then, illustrates two very useful ways in which a matrix 
can be expressed as a type II supervector. 


5.10 The Equality of Supermatrices 


If you look back at Eq. (5.9.6) in Section 5.9, you see that it violates the 
rule for equality which we gave for simple matrices. There we said that 
two matrices are equal if and only if each element of one was equal to the 
corresponding element of the other. This rule implied, of course, that the 
two matrices were of equal order, because otherwise, for each element in 
one, there could not be a corresponding element in the other. But you will 
see that in Eq. (5.9.6) the two matrices on the right which are said to be 
equal to each other do not even have the same order. The first is a column 
supervector while the second is a row supervector. Since supervectors are 
special cases of supermatrices, you can see that two supermatrices may be 
equal even though they do not have the same super order. In any case, 
however, two supermatrices are equal if they have the same order and if 
each submatrix in one is equal to the corresponding submatrix in the other. 
For submatrices to be equal, they must of course be of the same order and 
their corresponding elements must be equal. 

In general, two supermatrices may be equal if they are of the same order 
even though their corresponding matric elements are not equal. This may 
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not be obvious at first, but you will readily perceive it if you look at Figs. 
5.10.1 and 5.10.2. 


a= 3 à» — [4 2 I1] 
4 GA 3 
ао = |7 аз =12 1 5 
8 9 4 6 
Fig. 5.10.1 
3 4 z a 
bu = b 1 bis у D 
2s l5 
im [р 4 ba = | 6 
Fie. 5.10.2 


Suppose now in Fig. 5.10.3 we let a be a supermatrix made up of the sub- 
matrices in Fig. 5.10.1 and b a supermatrix made up of the submatrices in 


Fig. 5.10.2. 
cs ап id eee be м 
ал аз bn б» 


Fia. 5.10.3 


Now let us substitute the right-hand sides of the equations in Fig. 5.10.1 
into the first matrix of Fig. 5.10.3. We also substitute the right-hand sides 
of the equations in Fig. 5.10.2 into the second, or b matrices of Fig. 5.10.3. 
The result is 


Fia. 5.10.4 


You see that the corresponding scalar elements for matrix a and matrix b 
аге identical. 

We can now state the necessary and sufficient condition for two super- 
matrices to be equal. Two supermatrices are equal if and only if their 
Corresponding simple forms are equal. Аз you see in Fig. 5.10.4, if the 
Partitioning lines were removed from the matrices, the resulting simple 
matrices would be identical. 
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5.11 Vectors Whose Elements Are Matrices 


We have considered vectors whose elements are also vectors, and we have 
seen that there are two general types. We may also have supervectors 
whose elements are matrices. These we call type IJI supervectors. The 
partial triangular matrices written in supermatrix form in Figs. 5.7.3 and 
5.7.6 are examples of supervectors whose elements are themselves matrices. 
You may wonder whether, if we have a vector whose elements are matrices, 
we would use the primed symbol when the submatrices are in a row and the 
unprimed if the submatrices are in a column. To be consistent with the 
practice used in the case of simple vectors, we would have to follow this 
rule. However, it is usually more convenient to let the simple order of а 
matrix determine whether to use the prime rather than the order of the 
supervector that represents the matrix. 

One interesting and common example of а type III supervector is а 
prediction data matrix having both predictor and criterion attributes. If 
such a matrix is partitioned between these two sets of attributes it becomes 


a second-order type III supervector. For purposes of computation and 
analysis it is often useful to regard it as such. 


SUMMARY 


1. Definition of a supermatrix: 
a. Definition of a simple matrix: a matrix whose elements are scalar 


quantities is called a simple matrix to distinguish it from a super- 
matrix. 


b. A supermatrix is one in which one or more of its elements are them- 
selves simple matrices. All matric elements in a given row must 
have the same number of simple rows and all matric elements in à 
given column must have the same number of simple columns. 


c. The order of the supermatrix is the number of matric rows and 
columns. 


Partitioned matrices: Supermatrices may be constructed from simple 
matrices by partitioning between certain rows and columns. 


The natural order of a supermatrix: This must be defined, if at all, in 
terms of the natural order of its simple form. 


4. Symmetric partitioning: 


a. This is accomplished by partitioning between the corresponding 
rows and columns. 


b. Only square matrices can therefore be partitioned symmetrically. 


5. Symmetrically partitioned symmetric simple matrices: 
a. All diagonal matric elements are symmetric simple matrices. 
b. Corresponding off-diagonal matric elements are mutual transposes. 


en 
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6. The general diagonal supermatrix: 


a. This may have, for matric diagonal elements, any kind of simple 
matrices, including vectors and scalar quantities. 
b. All off-diagonal matric elements will be null simple matrices. 


- The partial triangular matrix as a supermatrix: This matrix may be 


partitioned to yield a triangular matrix and a rectangular matrix. 


- Type I supervectors: 


10. 


п 


а. А partitioned simple vector will be called a type I supervector. 
b. A column type I supervector has simple column vectors for elements. 
с. А row type I supervector has simple row vectors for elements. 


Matrices expressed as supervectors with vector element: 
а. Such vectors will be called type II supervectors. 
b. A matrix may be expressed as a column supervector whose elements 


are row vectors. 
с. A matrix may be expressed as a row supervector whose elements are 


column vectors. 

Equality of supermatrices: 

а. Two supermatrices are equal if and only if their simple forms are 
equal. 

Type III supervectors: 

а. Vectors one or more of whose elements are matrices will be called 


type III vectors. 
b. A type III supervector may be constructed by partitioning a simple 
matrix either between rows or between columns but not both. 


EXERCISES 


Given the supermatrix with matric elements 


a аш аз 
ад аљ аз 
аз а» аз 
ап a аз 


Assume: ац is [1 by 2], az and а are of widths 3 and 4 respectively; аз, аз, 
and as are of heights 3, 2, and 1 respectively. What are the orders of аг, as, аз, 
and as respectively? 


- What does the order of a supermatrix tell us about the order of its corresponding 


simple matrix? 


- Suppose we have an [8 by 6] simple matrix. How may we partition it to have а 


[2 by 3] supermatrix all of whose matric elements have the same simple order? 


+ If a matrix has been symmetrically partitioned, what do we know about its 


Simple order? 
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iE 


. Corresponding null submatrices above and below th 
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Given the symmetrically partitioned symmetric simple matrix 


Qi а аз 
а= | аһ an аз 
аз ар аз 
" 5 Е ^ 
(a) What kind of matrices are ai, a», аз? 


(b) What are the relationships between ai» and an, ai; and ам, аз and аз? 


. What is the meaning of the primes below the diagonal in the diagonal super- 


matrix? 
Dy 0 0 
Du Dp 0 
0 0' D; 


. If the subscripts in the super identity matrix below indicate order, what is the 


simple order of the matrix? 


I, 0 0 

Iz|0 n 0 

D^. 10/5 

. Given the matrix 
a 4 7.36.58 
rae 8 fe £ 
0 0 5 4.9 
(а) How would you partition it so that one of the submatrices is a triangular 


matrix? 
(b) Is it in the natural or transposed order? 


- If V and V’ are type I supervectors, what are (a) the elements of V, and (b) the 


elements of V’? 
If (b) is a type II row supervector, what are its elements? 


If (b) is a type II column supervector, what are its elements? 


ANSWERS 


- [3 by 3], [2 by 4], [2 by 3], (1 by 4] 
. Nothing 


. Partition between the 2nd and 3rd and the 4th and 5th columns, and between 


the 4th and 5th rows. 


. It is square. 


. (a) They are all symmetric. 


(b) One is the transpose of the other. 


e diagonal have reversed 
orders. 


10. 
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. (b) 


(a) Between 3rd and 4th columns 
(b) Transposed order 


- (a) Column vectors 


(b) Row vectors 


Column vectors 


. Row vectors 
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Chapter 6 


The Transpose 
of a Supermatríx 


6.1 General Matrix with Matrix Elements 


You can readily deduce the rules for finding the transpose of any super- 
matrix simply by observing what happens to the submatrices of a simple 
matrix when it is partitioned. Look at Fig. 6.1.1. 


Fic. 6.1.1 


Here we start with а 6 by 4 simple matrix and partition it into a 3 by 2 


supermatrix. Then we take the transpose of the matrix A in Fig. 6.1.1 and 
get à matrix as in Fig. 6.1.2. 


А' = 
1 3 4 "T 2 
8 T 6 3 5 8 
Fic. 6.1.2 


Suppose now we represent each of the submatrices in Fig. 6.1.1 by letter 
symbols with double number subscripts as in Fig. 6.1.3. 


82 
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1 6 1 8 
аа=|2 3 ae =|3 7 
4 5 4 6 


ИГЕ: uc pis 
Е e m s 


an = [3 1] az = [2 8] 
Fig. 6.1.3 


Thus we can write the matrix A in terms of the letter elements that stand 
for the matrices in Fig. 6.1.3. This gives us Fig. 6.1.4. 


ап аш 
А = |а ax 
аз а32, 

Fic. 6.1.4 


Next let us take ће transposes of the matrices in Fig. 6.1.3. This gives us 
the matrices in Fig. 6.1.5. Now compare the submatrices in Fig. 6.1.5 with 
those in Fig. 6.1.2. 


ber eor d su 2 
= |63 5 w= ls тв 
? 8 ; 9 7 
eT e x eu (qu Б 


, 3 , 2 
<< ff 


Fig. 6.1.5 


You will see that the three matrices on the left in Fig. 6.1.5 are respectively 
the three submatrices in the first matric row of the matrix in Fig. 6.1.2. 
Similarly, you will see that the three matrices on the right side of Fig. 
6.1.5 are respectively the three submatrices in the second matric row of the 
matrix in Fig. 6.1.2. We can therefore write the matrix in Fig. 6.1.2 in 
terms of letter elements with double subscripts as indicated in Fig. 6.1.6. 
+ + Li 
a а; 
A? = [ n n | 
Q 0» Age 
Fig. 6.1.6 


Rules for the transpose of a supermatrix 


If you now compare Figs. 6.1.4 and 6.1.6, you can easily discover the 
rules for writing the transpose of a supermatrix whose elements are letters 
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with double number subscripts. First, the order of the supermatrix is 
reversed. For example, Fig. 6.1.4 is a 3 by 2 supermatrix, whereas Fig. 
6.1.6 is a 2 by 3 supermatrix. Second, a row of the original matrix becomes 
a column of the transposed matrix, or, what amounts to the same thing, a 
column of the original matrix becomes a row of the transposed matrix. 
Third, as a consequence of writing rows for columns and vice versa, the 
order of the subscripts is changed, so that in the transpose the first subscript 
indicates the column in which the matric element is found, and the second 
element indicates the row. These three rules for writing the transpose of a 
supermatrix whose elements are letters with double subscripts are the same 
as those for the transpose of a simple matrix. A fourth rule, not the same, 
is that each element in the transpose is written with a prime, as in Fig. 6.1.6. 


We can incorporate these general principles for any order of supermatrix 
in Figs. 6.1.7 and 6.1.8. 


Qn аю ... Aim 
A = а Ax „яз. SUAE 
Qni Anm ... аһ, 
Fic. 6.1.7 
, , , 
Qu ар ... ац 
, , F] 
А’ = |2 02 ... а 
з з RU. 
Aim Geom ... аһ, 
Fic. 6.1.8 


In Fig. 6.1.7 we have an n by m supermatrix. In Fig. 6.1.8 we have the 
transpose of this supermatrix, an m by n supermatrix obtained from Fig. 
6.1.7 by writing rows for columns and taking the transpose of each of the 
elements. To summarize, then, the transpose of a supermatrix is obtained 
by writing rows for columns and taking the transpose of each element. 
You will see, of course, that Figs. 6.1.7 and 6.1.8 could also illustrate 
the transpose of a simple matrix, because a simple matrix is actually а 
special case of a supermatrix, in which the elements are scalars rather than 
matrices. As you recall, the transpose of a scalar is the scalar itself, so that 
Fig. 6.1.8 could represent a simple matrix. In that case, there wond of 


course be no point in adding the primes to the elements, since they would 
be equal to the original scalar elements. j 


6.2 The T n c 
fuum КС of a Symmetrically Partitioned Symmetric Simple 


Now let us see how the rules for getting the transpose of a supermatrix 
work out for а supermatrix that has been obtained from a symmetrically 


THE TRANSPOSE OF А SUPERMATRIX 85 


partitioned symmetric simple matrix. As you recall from Section 5.5, if we 
write the submatrices of such a matrix as letter elements with number 
subscripts, we have Eq. (6.2.1). 


Qi Gig ... Qim 

" 
Gis. De sau dw (6.2.1) 
Am Am +++ Umm 


Here we have a supermatrix of order m whose infradiagonal elements are 
the transposes of the corresponding supradiagonal elements and, as you 
recall, the diagonal elements are themselves symmetric matrices. Now let 
us write the transpose of a in Eq. (6.2.1) by writing rows for columns and 
writing the transpose of each element. This gives us Eq. (6.2.2). 


ay (а) ... (aw) 
ГА , СА + 
dz а аз» (аы) (6.2.2) 
fin Чы 25 бы 


Notice that the infradiagonal elements of Eq. (6.2.2) are exactly the same 
as the corresponding elements in Eq. (6.2.1). However, the diagonal 
elements of Eq. (6.2.2) are the transposes of the diagonal elements in 
Eq. (6.2.1). Also, the supradiagonal elements in Eq. (6.2.2) have the same 
subscripts as the corresponding elements in Eq. (6.2.1), but they are 
modified by the addition of parentheses and primes. Remember, however, 
that the diagonal matric elements of a symmetric matrix, symmetrically 
partitioned, are symmetric matrices and thus are not altered by transposi- 


tion. Therefore we have 
ay = ап аз = Are etc. (6.2.3) 


Recall also that the transpose of a transpose is the original matrix. There- 
fore we write 

(aig)! = аг (ai)! = аз (aij! = ai (6.2.4) 
If now in Eq. (6.2.2) on the right we substitute the right-hand side of 
Eqs. (6.2.3) and (6.2.4), we get Eq. (6.2.5). 


ап G2 ... аһ 
A 
аз аљ  ... Qm 
fa (6.2.5) 
+ + 
Aim Am +++ Omm 


But you will see that the right-hand side of Eq. (6.2.5), the transpose, is 
exactly the same as the right-hand side of Eq. (6.2.1), the supermatrix 
itself. This is, of course, as it should be, since we started with a symmetri- 
cally partitioned symmetric simple matrix. 
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We have then the simple rule that the transpose of a symmetrically 
partitioned simple symmetric matrix is the matrix itself. You might have 
seen this at a glance without going through all the steps given in Eqs. 
(6.2.1) through (6.2.5). However, these steps show how, by means of 
simple rules and well-defined symbols, we can arrive at certain conclusions. 
Sometimes the conclusions are very obvious; the use of rules and symbols 
seems unnecessary. At other times the conclusions are not easily drawn, 
but they can be arrived at by careful use of the 5 


ame simple rules and 
symbols. 


6.3 The Transpose of a Diagonal Supermatrix 
The simple diagonal matrix 


At this point in your study of matrices, it should be very easy for you 
to see that the transpose of a symmetrically partitioned diagonal matrix 
is simply the original diagonal supermatrix itself, Nevertheless, let us 
apply the simple rules we have learned and use letter symbols to prove this 
fact. We let Eq. (6.3.1) represent the diagonal supermatrix. 


d; 


(6.3.1) 
а, 
Each of the elements in the right side of this equation is itself a diagonal 


matrix. To find the transpose of D here, we simply write rows for columns 
and take the transpose of each element, This gives Eq. (6.3.2). 


(6.3.2) 
d! 
that the transpose of a di 
at we can write 


But now we remember 


agonal matrix is the diago- 
nal matrix itself, so th 


d = di «=й ete (6.3.3) 
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to this point, you have proved that the transpose of a diagonal supermatrix 
is the diagonal supermatrix itself. Note, however, that if the diagonal 
elements in Eq. (6.3.1) were other than diagonal simple matrices, we could 
not remove the primes on the right of Eq. (6.3.2). 


The scalar and the identity matrix 
Special cases of the diagonal supermatrix are the scalar supermatrix and 
the identity supermatrix. Since it is obvious that the transpose of each of 
these is the matrix itself, we need not go through the proof. We show that 
the transpose of а super identity matrix is equal to the matrix itself by 
Eq. (6.3.5). 
1, f 1, 
I; = 1» (6.3.5) 


Tn E 7 


6.4 The Transpose of a Type | Supervector 


You recall that in partitioning a simple column vector we get a column 
Vector whose elements are column vectors, and in partitioning a simple row 
Vector we get a row vector whose elements are row vectors. Suppose V isa 
column supervector obtained from partitioning a simple column vector as 
in Fig. 6.4.1. 


3 
2 
1 
V= 4 
6 
5 
Fig. 6.4.1 


Then its transpose would be given as in Fig. 6.4.2. 
V=(8 2 |14 | s 
Fic. 6.4.2 
If we let the subvectors be represented as in Fig. 6.4.3, then Figs. 6.4.1 and 
6.4.2 may be written as letters as in the left and right, equations of Fig. 
6.4.4, respectively. 


Ес. 6.4.3 
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yi 
у= | у, Via VE Vi 
Vs, 


Fic. 6.4.4 


Or, if there are n subvectors in V, we may indicate the supervector and its 
transpose by Fig. 6.4.5. 


Vi 


va|" = (И VE ж УЧ 


Fra. 6.4.5 


Thus we use the same rules to get the transpose of a type I supervector as 
we do to get the transpose of a general supermatrix. We write row for 
column and prime the elements. We cannot reverse the subscripts, of 
course, since each subvector has only one subscript. 


6.5 The Transpose of a Simple Matrix Written as a Type Il Supervector 
The column type Il supervector 


You recall that we can write a matrix as a column supervector whose 
elements are row vectors, or as a row vector whose elements are column 
vectors. Let us discuss first the column supervector whose elements are row 


vectors. Using the method given in Section 5.9, we write this as in Eq. 
(6.5.1). 


ai, 
а, 
Ga РЕ (6.5.1) 
(eal 
This is the way we indicate a matrix of order n by m if we want to write it 
as a column supervector with the rows of the matrix as the row subvectors. 


Now to indicate the transpose of a, we simply write row for column and 
take the transpose of the elements, as in 


а —[(a)' (a) ... (а), (6.5.2) 
But remember that two primes cancel each other. 


: "Therefore, in terms of 
the notation used for the row vectors of a matrix, 


we would have 


(à) —a. (шу=а, (an) = an. (6.5.3) 
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We could write a single equation for Eq. (6.5.3), using the first subscript as 
7 to show that it means any element of the supervector, as in 


(aj) = а. (6.5.4) 


If then we substitute the right side of Eq. (6.5.3) or Eq. (6.5.4) for the 
elements in the right side of Eq. (6.5.2), we have 


, 


С А Я (6.5.5) 


You see now that in Eq. (6.5.5) any element а;, is a row of the original 
matrix, written in column form in the transpose. 


The row type Il supervector 


Now consider a matrix written as а supervector whose elements are 
column vectors. This, as shown in Section 5.9, can be written as 


gE fær ба c Gala (6.5.6) 


Therefore to write the transpose of a given by Eq. (0.5.6), we write the row 
of elements on the right as a column and prime each of these elements, as 


in Eq. (6.5.7). 


a’ =|°? (6.5.7) 


On the right side of the equation, we now have the column vectors of the 
matrix а written in row form. We can now say that a’; is the ith column 
(that is, any unspecified column) of a written in row form in a’. 

Thus Eqs. (6.5.5) and (6.5.7) show that there are two different ways to 
write the transpose of a matrix as a supervector. Therefore we can write 
Eq. (6.5.8). 

ay 
$ 
a ea. m and akal (6.5.8) 


" 
@.т_|һ 


The meaning of the notation 


The meaning of all the symbols used in Eqs. (6.5.1), (6.5.5), (6.5.6), and 
(6.5.7) must be perfectly clear to you if you are to understand the rules we 
develop in the following chapters. Let us enumerate the specific things 
you should remember about the notation used in these equations. These 
àre as follows: 


(1) a is usually, but not necessarily, a vertical matrix. 
(2) a' is usually, but not necessarily, a horizontal matrix. 
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(3) a’ is always the transpose of a. 

(4) a is always the transpose of a’. 

(5) a can be expressed as either a column supervector with row elements 
or a row supervector with column elements. 

(6) a’ can be expressed as either a column supervector with row elements 
or a row supervector with column elements. 

(7) a. is always a row vector, each element of which is the corresponding 
element of the ith row of a. 

(8) a; is always a column vector, each element of which is the correspond- 
ing element of the ith row of a. 

(9) a.;is always a column vector, each element of which is the correspond- 
ing element of the ith column of a. 

(10) a‘, is always a row vector, each element of which is the corresponding 
element of the ith column of a. 


(11) а; and a’; are always row vectors even though a’ may be a vertical 
matrix. 


(12) a;, and ау are always column vectors even though a may be a 
horizontal matrix. 


| (13) The letter m or n means the number of rows in the matrix if it is 
inside column brackets. 


| (14) The letter m or n means the number of columns in the matrix if it 
is inside row brackets. 


(15) The letter m or n means the number of columns in the matrix if it is 
outside the column brackets. 


(16) The letter m or л means the number of rows if it is outside the row 
brackets. 


To summarize the supervector notation for а simple matrix and its 
transpose, we have then the very important equations (6.5.9) and (6.5.10). 


ai. 
а 
a= 2. =[a1 аз ... as]. (6.5.9) 
as. 8 
+ 
а.1 
ГА 
а= (а. о. ... а), = | (6.5.10) 
У 


Ве sure you understand exactly what these equations mean. They are 
fundamental to most computational procedures involving matrices, whether 


desk caleulators are employed or whether programs are written for high- 
speed electronic computers. 
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6.6 Тгапѕроѕе of a Type lll Supervector 


As you have learned (Section 5.11), а type III supervector is a special 
case of a supermatrix with only one row or one column of matric elements. 
Therefore the transpose of a supervector with matric elements introduces 
nothing new. We simply apply the rules of writing row for column or 
column for row and priming the elements. (Since for a type III supervector 
we use the prime in accordance with the simple order of the matrix rather 
than the order of the supervector, we do not necessarily write the row form 
with a prime.) If we let a supervector a be represented by Eq. (6.6.1), 
where a; and a; are matrices, 


a= [а a] (6.6.1) 
then the transpose, a’, would be given by Eq. (6.6.2). 
+ 
а = Н (6.6.2) 


On Ње other hand, if a supervector b were in column form with matric 
elements such as those in Eq. (6.6.3), then the transpose, b^, would be given 


by Eq. (6.6.4). 
= M (6.6.3) 


b = [bi b] (6.6.4) 


A special and frequently used example of a supervector with matric 
elements is a partial triangular matrix. Suppose we have an upper partial 
triangular matrix as in Fig. 6.6.1. 


324 5 2 

T'2|0 6 1 3. 5 

002 1 6 

Fig. 6.6.1 
Let us then define submatrices as in Fig. 6.6.2. 

32 4 5 2 
#=|0 6 t а=|8 5 
00 2 1 6 


Fic. 6.6.2 


Now, because ¢’ here is simply a matrix of the first three columns of Fig. 
6.6.1, and a is a matrix of the last two columns of Fig. 6.6.1, we can write 


T=( | al (6.6.5) 
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Then the transpose of 7" in Eq. (6.6.5) is simply Т, or is given by Eq. (6.6.6) 


T= 9 (6.6.6) 
a 


In Eq. (6.6.6), we have simply written column for row from Eq. (6.5.15) 
and have taken the transposes of the two matric elements. If we take the 
transposes of the matrices in Fig. 6.6.2, we have Fig. 6.6.3. 


8 00708 ves lese d 
1= 12 6 0 к= КА 
d peo 
Fic. 6.6.3 


If now we substitute in Eq. (6.6.6) the right-hand side of the equations 


given in Fig. 6.6.3, we have Fig. 6.6.4. This is obviously the transpose of 
Fig. 6.6.1. 


зоо 
2 0 
T= &- b 2 
a x1 
2.5 6 
Fic. 6.6.4 


Several important points should be noted in Eqs. (6.6.5) and (6.6.6). 
We are at liberty to use the prime or not, as we please, with any matric 
element, or any symbol for a type III supervector. However, once we have 
decided which symbols to prime and which to leave unprimed, then in the 
transpose of the supervector, we must remove the primes from the symbols 
that were primed and prime the symbols that were unprimed. 

You will note that these conclusions were anticipated in Section 5.7. 


SUMMARY 


- Transpose of the general supermatrix with matric elements: 
а. Rows are written as columns or columns as rows. 
b. Subscripts of matric elements are reversed. 
€. Each matric element is primed. 


- Transpose of symmetrically partitioned symmetric simple matrix: This 
must yield the same supermatrix when transposed. 


S 


Transpose of a diagonal supermatrix: 


a. The general case. The diagonal matric elements are transposed. 


b. If simple diagonal matrices comprise the diagonal matric elements, 
the transpose is the same as the original. 


THE TRANSPOSE OF A SUPERMATRIX 


- Transpose of type I supervectors: 
а. The column supervector becomes a row with row vector elements. 
b. The row supervector becomes a column with column vector elements. 
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Transpose of simple matrix expressed as type II supervector: 

а. The column supervector becomes a row whose row vector elements 
are transposed to column vector elements. 

b. The row supervector becomes a column whose column vector elements 


are transposed to row vector elements. 


Transpose of type III supervector: 


a. The column vector becomes a row with matric elements transposed. 
b. The row vector becomes a column with matric elements transposed. 


EXERCISES 


‚ If a, has the sequence of elements 4, 6, 7, 


- If b’, has the sequence of elements 7, 


- If cf, has the sequence of elements 


6, 7,5 
аз. has the sequence of elements 3, 9, 2, 8 
аз, has the sequence of elements 5, 6, 4, 2 


write a’. 


3, 6, 9 
b^; has the sequence of elements 8, 2, 4, 2 
b^; has the sequence of elements 4, 1, 9, 7 
write b. 


7, 4, 6, 
сї. has the sequence of elements 9, 3, 2 
с}. has the sequence of elements 3, 9, 5, 


write c. 
If fı has the sequence of elements 6, 4, 3 


Í.: has the sequence of elements 5, 2, 1 
f.s has the sequence of elements 9, 3, 8 


write f’. 


Let A be a zero-one matrix where the entities are five rats, the attributes four, 


and the elements successes or failures, thus, 


нн о н 
= © = © 


0 
0 
1 
1 


со 


Write out each of the following vectors: (a) Aj. 


()4A^5 (fA. 


н юе с о 


(b) А, 


(с) As. (d) Af, 
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6. Let B be a zero-one matrix where the entities are four people, the attributes five 
test items, and the elements success or failure on the item, thus 


1 
1 
1 
1 


-Omm 
—-——o- 
orom 


1 

1. 

0 

0 
Write out each of the following vectors: (a) B. 
(e) Bi. (f)B.s. 


з» (B. (0) Ва (A) BS. 


7. Indicate whether each of the following in simple form is a matrix or a vector: 


11. т. (е) Га. xe Д Ta 
t 
wle] w| up 
d; an f. 
@) fea 22 6. 8] — (D zo. cl] T4 
+ 
(g) r2 
Uus 
T.r 
(h) [z^ 1, nn 4] 
ANSWERS 
i. 4.8.5 2. 7874 3. 7465 
q.2|99 6 ре 9-2 c=|9 3 2.1 
7T 24 649 BOF pm 
5 8 2 9.2 5 
e 64 3 5. 1 (b A4, =(1 1 1 1 1 
fi=|5 21 0 
938 sa) Am | 
T 
0 (d) 4,=(0 01 d (л-р 010 1] 
(с) А, = 1 
1 
(f) Ag = 


1 6. (а) Bs =[1 0 1 0 
1 
(b) В, = 


онон н 
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( Bs 


7. (a) matrix (b) vector 
matrix (h) vector 


(d Bj = п 0 


(c) matrix 


1 


0. 3] 


(d) vector 


(е) В = 


(е) matrix 


К! 


(f) vector 


(g) 
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Chapter 7 


Addítíon and Subtractíon 
of Matríces 


7.1 Addition of Simple Matrices 


The sum of two matrices 


As in arithmetic and ordinary algebra, so also in matrix algebra we have 
the operation of addition. In arithmetic any two numbers can be added 
together whether they are positive or negative. However, two matrices 
can be added if and only if they have the same order. The reason for this 
will be clear when we define the sum of two matrices. The sum of two 
matrices is a matrix, each element of which is the sum of corresponding 
elements of the two matrices being added together. 

The sum of two vertical matrices is shown in Fig. 7.1.1. 


4 6 3 2 443 6+2 7 8 

з 7/+/6 4|=|8+6 74+4/=]9 11 

2 5 3 6 2+3 5+6 5 11 
Fig. 7.1.1 


The sum of horizontal matrices is shown in Fig. 7.1.2. 

6 2 4 2 5 8]_[6+2 2+5 JHF- 7 12 
397|*|[7 4 3] 7 [3+7 9+4 7+3 10 13 10 
Fig. 7.1.2 
We can also add a diagonal matrix to a square matrix of the same order, 

as in Fig. 7.1.3. 
400 6 3 2 4-46 3 2 103 2 
03 00|l435 4|=| 3 3+5 4 |-| 3 8 4 
2 4 € 2 4 5+7 2 4 12 


0 0 5 
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Since vectors are special cases of matrices, they are added in the same way. 
Figure 7.1.4 shows the sum of two third-order column vectors. 


4 3 4+3 7 

ЗИРЕ 

2 T 2-7 9 
Fig. 7.1.4 


From the definition of the sum of two matrices, it is obvious that the 
order of a sum of two matrices is the same as the order of the matrices that 
were added together. 


The sum of any number of matrices 


Аз you may have guessed, we can take the sum of any number of ma- 
trices. However, all of those to be added must have the same order. Each 
element in the sum is the sum of the corresponding elements of the indi- 
vidual matrices that are added. Figure 7.1.5 shows the sum of three 
second-order matrices. 


Г H+? +? = 6+2+7 5+4+9] [15 | 
4 3 85 6 4] [44346 345-4|^7]|13 12 
Fig. 7.1.5 


In general, suppose we have matrices a,b,...zof ordern by m defined by 


Eqs. (7.1.1), (7.1.2), and (7.1.3) respectively. The sum of these matrices is 
then as given in Eq. (7.1.4). 


ал а аљ 

а= |910 Gm sis Gin (7.1.1) 
ho Ae io. а 
bu by bim 

p= |dan be bom (7.1.2) 
he УОИ 
211 212 Zim 

PONE A Zom (7.1.3) 
Re P 


qp pev dares 
ац а ЧЕ аә + be + + 2 аһ + b, 
ТР ж m +... + Zim 
а + ba +... а ав dU» +... + г ios dis beoe db ps 
ded LL ci ы үүн 
(7.1.4) 
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7.2 Some Simple Laws of Addition 


We shall now give two simple laws of addition so familiar and obvious 
that you may wonder why we bother to state them. The reason will become 
evident later, when you find that one of the laws you have taken for granted 
in arithmetic and scalar algebra does not hold for matrix algebra. 


The associative law 


The first law is the associative law of addition. An example of the associa- 
tive law in arithmetic is given by the simple equations in Fig. 7.2.1. 


10 
10 


2+3+5 = (2+3) +5 = 5 +5 
24+3+5=2 + (3 +5) = 2+8 
Fig. 7.2.1 


You see that in the first equation in Fig. 7.2.1, we added the first two 
numbers, 2 + 3, to get 5, and then added this sum to the last number, 5, 
to get 10. In the second row, we first added the last two numbers, 3 + 5, 
to get 8, and then added the first number, 2, to 8 to get 10. If we take a 
little more complicated example, where we have four numbers in the sum, 
there are still more ways in which we can get the sum of the four numbers, 


as shown in Fig. 7.2.2. 


(4+1)+2+6=(5+2)+6=7+4+6=13 

4+(1+2)+6=44+(8+6) =4+9=18 

4+14+(24+6)=(44+14+8=5+8=18 

(4+1) +2+6=5 + (2+6) = 5 +8 = 13 
Fia. 7.2.2 


In general then, if we have three or more elements to be added, we can 
reduce the total number of elements to be added by one if we first take the 
sum of any two adjacent numbers. From this new series, we can further 
reduce the number to be added by one if again we add any two adjacent 
numbers. We can continue this process until we get the sum of all of the 
numbers. The associative law says that irrespective of which two adjacent 
elements we take first, which two second, and so on, we will always get the 


same total for all the elements. | | | 
The associative law of addition for matrices can be illustrated if we define 


matrices a, b, and c, as in Fig. 7.2.3. 


eg hdg üt 


Fig. 7.2.8 
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Now Figs. 7.2.4 and 7.2.5 show how the associative law of addition applies 
for the sum of the matrices a, b, and c in Fig. 7.2.3. 


3+2 4+1 sal 57:5 ED Ee 
iwi ee р NE AE 12 


Fig. 7.2.4 
DEE 2+3 142] [34 SS Te? 
a+b+e= |2 "e Fen УЫ НЕЕ е | 
Ета. 7.2.5 


In Fig. 7.2.4, we added first а and b, and then to this sum we added с. In 
Fig. 7.2.5, we added first b and c, and then added this sum to a. The results 
are of course the same as you would expect them to be in arithmetic. The 


associative law of addition for three numbers or matrices can be stated in 
the form of Eq. (7.2.1). 


(a+b) +с=а+ (+ о) (7.2.1) 


As a child studying arithmetic, you learned to use the associative law of 
addition to check your sums. For example, if you had a column of numbers 
as in Fig. 7.2.6, you learned to get the sum as in Fig. 7.2.7. 


[eoo 


Fia. 7.2.6 


(8+5) +7 +4 = (8+7) +4 = 1544 = 19 
Fig. 7.2.7 


Then you would check this sum as in Fig. 7.2.8. 
З+5+ (7+4) =3 + (+411) = 3 +16 — 19 
Fic. 7.2.8 


As a third-grader, however, you would have been utterly bewildered if the 


teacher had told you to check your sum by using the associative law of 
addition. 
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The commutative law 


A second simple law of addition is the commutative law. Ап example of 
the commutative law in arithmetic for addition of two numbers is given in 
Fig. 7.2.9. 

3+4=7 a 
Fic. 7.2.9 


For three numbers, an example of the commutative law of addition is given 
in Fig. 7.2.10. 


20-543 
5+3 + 2 


10 
10 


10 2+3 +5 = 10 5+2+3 
10 3+5 +2 = 10 30-246 


Fig. 7.2.10 


ll 


Here we have six different ways in which to arrange the order of the 
numbers 2, 5, and 3. No matter in what order we arrange them, they add 
up to 10 in every case. Here again, this seems elementary, but when we 
take up the multiplication of matrices, you will learn that the commutative 
law does not hold. Figures 7.2.9 and 7.2.10 show how the commutative law 
of addition holds for scalar quantities. From the rule for the addition of 
matrices, it should be easy to see that the commutative law for the addition 
of matrices will hold also. Figures 7.2.11 and 7.2.12 illustrate the commu- 
tative law for the addition of matrices. 


3 4 бз]. [apa ЕЕ i 
4 e|l*[2 5] "142 6-5] [6 1 


Fie. 7.2.11 


2.41 з 4] [2-8 EE 1 
5 51+ [4 6] = [2+4 5-6] 16 n 
Fig. 7.2.12 


the left side of the equation the same two matrices 
The only difference is that the order of the two 
matrices is reversed. Nevertheless, as any child can plainly see, the sum is 

h of the two possible orders they are added. 


the same, no matter in whic 
The commutative law of addition for two matrices can be expressed as 


In Fig. 7.2.12, we have on 
we have in Fig. 7.2.11. 


at+tb=b+a (7.2.2) 


This equation holds whether we are talking about scalar quantities or 
matrices. For three scalars or matrices, the commutative law of addition 


is given by the six sets of equations in Fig. 7213: 
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ate+b=e b+a+c=e 
e c+a+b=e c+b+a 


Fic. 7.2.13 


a+b+e 
beta 


І 
^ 


ll 
c 


We can then summarize the commutative law of addition by saying that 
no matter in what order a set of matrices is arranged, their sum will always 
be the same. 


7.3 Subtraction of Simple Matrices 


As you may well suppose, the subtraction of one matrix from another 
follows the same principle as the addition of two matrices. First, а neces- 
sary and sufficient condition that one matrix can be subtracted from 
another is that the two matrices have the same order. Second, each element 
of the difference matrix is the difference between the corresponding elements 
of the two matrices. An example of how one 2 by 3 matrix is subtracted 
from another 2 by 3 matrix is shown in Fig. 7.3.1. 


HE ее а фео SB) (poc 4 
92 11 So 3 D] [018 Ош ap к md 
Fig. 7.3.1 


In general then, we can illustrate the difference between two n X m 
matrices, using letter elements, by Fig. 7.3.2. 


bu 6... Dim Cu Cie 


Cim 
ba бш... bom [1 €» Com 
Ва Dag. ass Бка Cni Сз Cnm 
bi — cy by — Ce... dim — Cim 
ba — cy be — бз. ... Dom — Com 
ba = Cn bno — ES сш» Bao. — Cnm 
Fig. 7.3.2 


7.4 Addition and Subtraction of Simple Matrices 


: Аз a scalar quantity may be the result of both addition and subtrac- 
tion, So a matrix may be made up of the sums and differences of a number 
of matrices. Each element of such a matrix would of course be made up of 


the sums and differences of the corresponding elements of the other ma- 
trices. An illustration is Fig. 7.4.1. 
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2.8 1 7 3 6 4 8 

[р J-i; 11- [8 Pr Е 
аИта [2.9 
5-4-7+7 7—3—5-48| |! 7 


Fig. 7.4.1 


We may appropriately regard subtraction as a special case of addition 
in which all of the signs of the elements of one of the matrices have 
been changed and the resulting matrix added to the other. Then it is easy 
to see that both the associative and the commutative laws of addition will 
hold for the sums and differences of a number of matrices. 


7.5 The Addition of Supermatrices 


From what you know about the sums and differences of simple matrices, 
you could readily make up corresponding rules for supermatrices. Thus the 
sum of two or more supermatrices is a supermatrix whose matric elements are 
the sum of the corresponding matrix elements of the individual matrices. 
The difference between two supermatrices is a supermatrix whose matric 
elements are the differences of the corresponding matric elements of the 
two matrices. 

So, again, we may have a supermatrix made up of the sums and differ- 
ences of any number of supermatrices. It is defined in the same way as the 
corresponding simple matrix, except that the elements are themselves 
matrices. 


Conformability requirements 


Supermatrices, like simple matrices, cannot be added or subtracted unless 
they have the same order. Even though supermatrices have the same order, 
however, it may still be impossible to add or subtract them. Not only must 
the order of the supermatrices be the same, but also the order of the 
corresponding matric elements of the supermatrices must be the same. 
This principle is illustrated in Figs. 7.5.1, 7.5.2, and 7.5.3. 


NI ZA 
О at a6 а = ү 


aa = [4 3] аз 
Fig. 7.5.1 


[ 
= 


bu = [7] bi = [1 3] 


7 4 1 
ъ=] ==[; ij 


106 SIMPLE MATRIX COMPUTATIONS 


Suppose we make up supermatrices from these figures as shown in Fig. 7.3.5. 


_ |% а» Б Fa “| 
тоё йа а» ba bo 
Fic. 7.5.3 


It is clear that a and b in Fig. 7.5.3 are both second-order square super- 
matrices, but here we cannot add together the corresponding matrix 
elements of a and b because they do not have the same order. For example, 
ay is à 2 by 2 matrix and by isa 1 by 1 matrix ог scalar, and similarly, ai» 1$ 
а 2 by 1 matrix whereas by isa 1 by 2 matrix. Therefore, it is clear that two 
or more supermatrices may be added or subtracted if and only if they are 
of the same order and their corresponding elements are also of the same 
order. (We sometimes call the order of a supermatrix the superorder.) 


7.6 The Transposes of Sums and Differences of Matrices 


The rules for transposes of sums and differences of matrices are very 
simple. The transpose of a sum of two matrices is equal to the sum of the 
transpose of the two matrices. Figure 7.6.1 is an illustration of this princi- 


ple. 
[| | +[1 1 е аара 
3 4 PERIA ee Е 
оро D ms 2d з 4 
Feige al alee EHA 
Fig. 7.6.1 
In symbolic form, this is 
lato = а p (7.6.1) 


The transpose of the difference between two matrices is the difference 
between their transposes. An illustration is Fig. 7.6.1. 


CAED- ЕЕН 
„ее е B 3-3 3] 


This rule is expressed symbolically as 


(a=b) =a -v (7.6.2) 
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'The transpose of a sum and difference of matrices is equal to the sum and 
difference of the transposes of the matrices. You can readily construct 
examples to illustrate this principle. The rule is 

(atb—c)/ =a +6 – с (7.08) 


The same rules that hold for the transposes of the sums and differences 
of simple matrices also hold for the sums and differences of supermatrices. 
You must remember, of course, that when you take the transpose of à 
supermatrix you not only write rows for columns, but you also take the 
transpose of each-matric element of the supermatrix. 


7.7 Sums and Differences of Special Kinds of Matrices 


The sums and differences of certain kinds of matrices have special 
properties that must be kept in mind. 


Diagonal matrices 
You see at once that the sums and differences of diagonal matrices are 
a diagonal matrix. If we let D with a subscript indicate various diagonal 
matrices of the same order, this principle can be readily stated. 
Da + Dy = De — De = D; (7.7.1) 


This is so obvious that no proof is needed. Since all nondiagonal elements 
are 0, their sums and differences are 0. 


Scalar matrices 
The sums and differences of scalar matrices are scalar matrices. Figure 
7.7.1 illustrates this principle. 
3 4 2 5 


3 4 2 5 
Fie. 7.7.1 


Symmetric matrices 
The sums and differences of symmetric matrices are symmetric matrices, 
as in Fig. 7.7.2. 


B x d 24 vk a 2-1 2 X1 
2 4 29|4|i 2 i|[2 2 Lif 42 
12 5 1I LL 12 6 
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Here you see that the matrix to be subtracted is the transpose of the first. 
You also see that in the difference matrix all the diagonals are 0 and 
corresponding nondiagonal terms are opposite in sign. 

We can easily generalize these two rules for any square matrix. First we 
write in expanded form the difference between a square matrix and its 
transpose as in Eq. (7.7.13). 


Cit Jd ams odis Qi dy ... am би C sss Cin 

Qn аљ ... Aan Om Gn o... Ua] |р Cm... C 

Wu ce oue Whey Gi» XR. vue Gum Си Cn2 „у. Cnn 
(7.7.18) 


In compact or symbolic notation, we can write the equation as 
@ cdm (7.7.14) 


For Eq. (7.7.14), we now want to prove that Eqs. (7.7.15) and (7.7.16), 
representing the two rules, are true. 


emu (7.7.15) 
бу = Sep (7.7.16) 


First we subtract each element in the second matrix on the left side of Eq. 


(7.7.13) from the corresponding element in the first matrix on the left of 
(7.7.13) and get Eq. (7.7.17). 


ап —ün Ge —ag ... An 


= Gay сп Cio D Cin 

Qn — ав ав – аљ ... аһ — Ang Cn. Ch ..„ Con 

Qni — Gin а» — Aen ny 7305 Cnt © was Can 
(7.7.17) 


If now from Eq. (7.7.17), we e 
column of the left matrix t 
the right matrix we get 


quate the element in the ith row and jth 
о the element in the ith row and jth column 1n 


Qij — ан = с, (7.7.18) 


If we equate the element in the jth row and ith column of the left matrix 
to the element in the jth row and ith column of the right matrix, we get 


ан — dij = сод (7.7.19) 
If we change the Signs of all terms in Eq. (7.7.19) we get 
=ü; + Qij = — Cji (7.7.20) 


Remembering that the commutative law holds for both addition and 
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subtraction, we can change the order of the terms on the left side of Eq. 
(7.7.20) to get 


ау — dj; = —6y (7.7.21) 


But notice that the left sides of Eqs. (7.7.18) and (7.7.21) are exactly the 
same. Therefore we can equate the right side of the two equations to get 


Cj = —6j (7.7.22) 


Now Eq. (7.7.22) is exactly the same as Eq. (7.7.16), the second rule for a 
Square matrix. Therefore Eqs. (7.7.18) to (7.7.21) prove that if from a 
Square matrix we subtract its transpose, we get a matrix in which the ijth 
element is the negative of the jith element. 

The proof is even simpler that the diagonal elements of the difference 
between a square matrix and its transpose are 0. If we equate the ith 
diagonal element in the left matrix of Eq. (7.7.17) to the ith diagonal 
element in the right side, we have 


а; — Qi = би (7.7.23) 
But any number subtracted from itself is 0; therefore, we have 
ai — ах = 0 (7.7.24) 


If now we substitute the right side of Eq. (7.7.24) into the left side of Eq. 
(7.7.23), we have, after transposing, Eq. (7.7.24), which we wished to prove. 


ci = 0 (7.7.24) 


Since this is exactly the same as Eq. (7.7.14), Eqs. (7.7.22) through (7.7.24) 
prove that the principal diagonal of the difference between a square matrix 


and its transpose is 0. 
SUMMARY 


1. Addition of simple matrices: 
a. The sum of two matrices is defined as a matrix each element of which 


is the sum of the corresponding elements of the two. If A + B = @; 
with elements 4;;, Bi; апа C;;, respectively, then А;; + By = 05. 
b. Therefore only matrices of equal order can be added together, and the 
order of a sum is equal to the order of the matrices. 
c. The sum of any number of matrices is a matrix each element of which 
is the sum of corresponding elements of all the matrices. 


2. Some simple laws of addition: 
а. The associative law: (a + b) + c = a + (b + c). 
b. The commutative law: a + b = b + a. 
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3. Subtraction of simple matrices: 
a. Subtraction may be regarded as a special case of addition where the 
signs of all elements in the matrix to be subtracted are reversed. 
b. Then both the associative and commutative laws of addition hold. 
4. Addition and subtraction: 


a. Any number of matrices may be combined by addition of some and 
subtraction of others. 


b. The associative and commutative laws hold for addition and sub- 
traction of any number of matrices. 
5. Addition and subtraction of supermatrices: 
a. The sum or difference of two supermatrices is a supermatrix each of 


whose matric elements is the sum or difference of corresponding 
elements of the two matrices. 


b. Supermatrices may be added or subtracted; therefore, if and only if 
they have the same superorder and corresponding matric elements 
are of the same order. 

6. The transpose of a sum or difference: 

a. The transpose of a sum or difference of simple matrices is equal to the 

sum of their transposes, that is, (a + b) = а + b. 


b. The transpose of a sum or difference of supermatrices is equal to the 
sum or difference of their transposes. 


7. Sums and differences of special kinds of matrices: 
a. The sum of any number of diagonal matrices is a diagonal matrix. 
b. The sum of any number of scalar matrices is a scalar matrix. 1 
с. The sum of any number of symmetric matrices is a symmetric matrix. 
d. The sum of a square matrix and its transpose is a symmetric matrix. 
e. The difference between a square matrix and its transpose is a skew 
symmetric matrix. 
EXERCISES 
1. lf 
а+ = с 
апа а is of order 40 by 3, 
(a) What is the order of 5? 
(b) What is the order of c? 
2. If 
a+b=f 


and 


10. 


1. 


12. 
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what law is illustrated by the following equation? 
atg-íctc 


Illustrate the commutative law of addition by writing the following equation in 
as many ways as possible: 


a+b+c=g 
If A is a supermatrix and 
A+B=C 
(a) what do we Know about B? (b) What do we know about C? 
Write 


f=(atb—-c)’ 
without the parentheses. 
If D, and D, are diagonal matrices and 
р. + D = С 


what kind of matrix is С? 


. If œ and @ are scalar matrices and 


а+В= 7 
what kind of matrix is y? 
If S, and S; are symmetric matrices and 
Sı — S: = b 
what kind of matrix is b? 
If S is a symmetric matrix and D a diagonal matrix, and 
р+8 = К 
what kind of matrix is К? 
If 
т+{+х' =y 
(а) What do you know about z? (b) What do you know about y? 


If 

r—a'-y 
what do you know about y? 
If Z is skew symmetric and 

2+2 = Ж 


what do you know about W? 
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13. If u is skew symmetric and 
и = и = М 


what do you know about M? 
ANSWERS 
1. (a) 40 by 3 (b) 40 by 3 
2. The associative law 


3.a+c+b=g b+a+ce=g b+c+a=g c+a+b=g 
c+b+a=g 


4. (а) The superorder of B is the same as A, and corresponding matric elements of 
A and B have the same order 


(b) The superorder of C is the same as A, and corresponding matric elements of 
A and C have the same order 
f -— a + b TEN с! 


mo EN 


А diagonal matrix 


x 


A scalar matrix 

8. A symmetric matrix 

9. A symmetric matrix 

10. (a) It is square (b) It is symmetric 
11. It is skew symmetric 
12. It is a null matrix 


13. M = 2u 


Chapter 8 


Vector Multiplication 


Like scalar algebra and arithmetic, matrix algebra allows not only the 
operations of addition and subtraction, but also the operation of multiplica- 
tion. Multiplication in matrix algebra, however, is not quite as simple as 
multiplication in scalar algebra. All multiplication in matrix algebra can 
be described in terms of the multiplication of vectors by one another. 
"Therefore, we shall first consider the multiplieation of vectors before we 
take up the multiplication of matrices in general. Two kinds of vector 
multiplication will be described, since each is convenient for defining 
matrix multiplication and for indicating computational procedures with 
both desk calculators and electronic computers. 

You must note, however, that our treatment of vector and matrix 
multiplication in this and the following chapter differs from the traditional 
mathematical presentation. Since matrix notation is much more economi- 
cal for the handling of behavioral science data than scalar notation we shall 
dispense as rapidly as possible with scalar notation and use it only rarely. 
The treatment in this and the following chapter is designed to help you 
take advantage as completely and quickly as possible of the simplicity and 
power of matrix notation. 


8.1 The Minor Product of Two Vectors 


The first kind of vector multiplication we shall consider is called the 
minor product of two vectors. We shall also refer to this product as their 
scalar product; the two terms will be used interchangeably. Figure 8.1.1 
shows the minor product of two third-order vectors. 


2 
[3 4 7]|6 = (3X 2) + (4 X 6) + (7 X 1) = 37 
1 
Ета. 8.1.1 
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The factors of a minor product 


We speak of the factors of a product in vector multiplication just as we 
speak of the factors in the product of scalar multiplication. "Therefore, in 
Fig. 8.1.1 we have two factors on the left side. The first of these factors 
is а third-order row vector and is called the prefactor. The second is a 
third-order column vector and is called the postfactor. From the right side 
of the equation in Fig. 8.1.1, you see that we have obtained the minor 
product of the two vectors by taking the product of corresponding elements 
in the two factors and adding these products to get a single number. Thus 
the first element in the first factor is 3 and the first element in the second 
factor is 2. The second element in the first factor is 4 and the second cle- 
ment in the second factor is 6. The third element in the first factor is 7 and 
the third element in the second factor is 1. The product of the two vectors 
is 37. 

You should notice then that in multiplication for a minor or scalar 
product of two vectors, the prefactor must be a row and the postfactor 
must be a column. Therefore, in speaking generally of the minor or scalar 
product of two vectors, we say that a row vector is postmultiplied by а 
column vector, or that a column vector is premultiplied by а row vector. 
The rule is that in а scalar product of two vectors, the prefactor must always 
be the row vector and the postfactor must always be a column vector. 


Notation for the minor product 


The minor product of two vectors may now be defined as the sum of the 
products of corresponding elements of the two vectors. In general, if the 
vectors V, and V, are given as in Fig. 8.1.2, their minor 
is defined by Eq. (8.1.1). 


or scalar product 


а bi 
al) ws 
" ch 
Fic. 8.1.2 
bi 
Киш desee e] | | mobi ads Булыйк; (8.1.1) 
b. 


From the definition of the minor Product of two vectors, it is clear that 


the minor product is always a scalar quantity. It is always a single number 
or letter that stands for a single number. Furthermore, from the definition; 


the vectors must be of the same order. The scalar or minor product can pe 
found for any two vectors of the same order. 
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8.2 The Transpose of a Minor Product of Two Vectors 


Since the minor product of two vectors is a scalar quantity, then as you 
would suppose, the transpose of the minor product of two vectors is the 
same scalar quantity. This can be expressed as 


(VaVe) = Vive (8.2.1) 


It is also easy to prove that the transpose of the scalar product of two 
vectors is equal to the product of their transposes in reverse order. This 
rule can be represented by 


Viv, = ViVa (8.2.2) 


Equation (8.2.2) says that if we postmultiply the row form of a V, vector 
by the column form of a V; vector, we get the same results as if we post- 
multiply the row form of the V; vector by the column form of the V, vector. 
This is very easy to prove from the rule of scalar multiplication of vectors. 
First we go back to Fig. 8.1.2 and write 


ViVa = [bi b . Dall |= bad bam +... + bnan (8.2.3) 


Now let us compare, term for term, the last expression (the one after the 
last equals sign) in Eq. (8.1.1) with the last expression in Eq. (8.2.3). Here 
we have a,b, for the first term in Eq. (8.1.1) as against bia; for the first 
term in Eq. (8.2.3). The second terms are respectively axb» and ba, and 
so on to the last term. In other words, the factors in the products of 
corresponding terms are simply reversed. But remember that each product 
has scalar factors, that is, it has simply the elements of the vectors as 
factors, As you well know, in scalar algebra, A X B = B X A, or in 
arithmetic 3 X 4 = 4 X 3. Therefore, the last expressions in Eq. (8.1.1) 
and Eq. (8.2.3) must be equal. If the last expressions are equal, the first 
expressions must also be equal; therefore we can write 


УУ» = ViVa (8.2.4) 


Now you can see that Eq. (8.2.4) is exactly the same as Eq. (8.2.2). We 
have therefore proved that the transpose of a minor product of two vectors 
is equal to the product of their transposes in reverse order. 

Although for demonstration we have proved this rule formally by means 
of equations, you can see that it must be true merely by using common 
sense. Let us look at the product of a row vector by a column vector when 
the elements are numbers as in Fig. 8.2.1. 
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11 


1 = z 
в 2 ale|-exn+exo+exa-3+12+10=25 
2 


Fic. 8.2.1 


Now look at the product of the transposes of the two vectors on the left in 
Fig. 8.2.1 in reverse order, as in Fig. 8.2.2. 


: 5 
п 6 а|:|-«хэ+вхә+@хы =з жш+ю=» 
5 


Fig. 8.2.2 


If you are familiar with correlation coefficients, you know that in age 
to calculate a correlation coefficient, we must get a sum of products i 
Scalar quantities which is usually written ZXY. Having learned how i 
take the scalar product of two vectors you can therefore write a sum of aa s 
products in vector notation. Suppose we have a column of x values npe 
column of y values for which we want to get the sum of cross products. 


We represent the columns of z and y values respectively as in Fig. 8.2-3. 
Ti Yı 
z-|^ y=| # 
2, V. 
Fic. 8.2.3 
If you have studied elementary statistics you know that EXY can be 
written as 


ZXY = XY, XO... XY, (8.2.5) 
But we can also write the scalar product of two vectors as 


RE = KY 4, S y (8.2.6) 
You see that the right side of Eq. (8.2.5) and Eq. (8.2.6) are exactly bel 
same. Therefore the left sides must also be exactly the same and we €& 
write 
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8.3 The Major Product of Two Vectors 


The second kind of product of two vectors we shall consider is one not 
commonly discussed in books on matrix algebra, but very useful in the 
behavioral sciences. We shall call it the major product of two vectors. A 
numerical example is shown in Fig. 8.3.1. 


2 o3 BS 2» 294 2 6 4 8 

alu 3 2 42|4X1 4X3 4X2 4x4|-|4 12 8 16 

3 3x1 8x8 3X2 3X4 3 9 6 12 
Fic. 8.3.1 


Now if you study this equation carefully, you will make a number of 
important observations. First, it is similar to the equation for the minor or 
scalar product of two vectors in that it has а prefactor and a postfactor. 
The vector on the left is the prefactor and the vector on the right is the 
postfactor. In the equation for the minor product of two vectors, how- 
ever, the prefactor was a row vector and the postfactor а column vector. 
For the major product of two vectors, we have just the reverse; the pre- 
factor is a column vector and the postfactor isa row vector. Therefore, to 
get the major product of two vectors we postmultiply a column vector 
by a row vector or we premultiply the row vector by a column vector. 

If you examine the matrices following the first and second equality signs 
in Fig. 8.3.1, you see that we have formed the major product of the two 
Vectors in a very simple manner. For the first row of the product matrix 
each element in the postfactor is multiplied by the first element in the 
prefactor. For the second row each element in the postfactor is multiplied 
by the second element in the prefactor. For the third row each element in 
the postfactor is multiplied by the third element in the prefactor. 


Notation for major product 


If we go back to the vectors Va and V» we can illustrate the general rule 
for getting the major product of two vectors by Eq. 8.3.1. 


а аһ d .. г 
Wie [и de м] уы үт, (8.3.1) 
а. аһ даф... аф», 


3.1) that the major product of two vectors is a 
matrix whose number of rows is equal to the order of the prefactor and 
Whose number of columns is equal to the order of the postfactor. Each 
clement of a row vector of the product matrix is obtained by multiplying 


each element of the postfactor by the element in the prefactor corresponding 
each column vector of the product matrix is ob- 


You see from Eq. (8. 


to that row. Similarly, 
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tained by multiplying each element in the prefactor by the element in the 
postfactor corresponding to that column. 

In general, if we indicate the major product of two vectors by the symbol 
C, we can write Eq. (8.3.1) symbolically as 


У.У = С (8.3.2) 


The element in the ¿th row and jth column of С is the product of the ith 
element of the prefactor and the jth element of the postfactor. Thus 


Ci; = аф; (8.3.3) 


Order characteristics of the major product 


It is clear then from the definition of the major product of two vectors 
that there are no restrictions on the orders of the two vectors. For the 
scalar product of two vectors, we know that both factors must be of the 
same order, so that corresponding elements can be multiplied together. 
For the major product of two vectors, there is no such necessity. But, 
while the minor product is always a scalar quantity, the major product of 
two vectors is never a scalar quantity but rather a matrix. The number of 
rows in this matrix is determined by the column vector, or prefactor, and 
the number of columns is determined by the row vector, or postfactor. 
Thus, in summary, the order of a matrix product of two vectors is given by 
the order of the factors and in the same sequence. For example, a fourth- 
order column vector times a third-order row vector gives a 4 by 3 matrix. 


8.4 The Transpose of a Major Product of Two Vectors 


We shall now prove that the transpose of the major product of two 


vectors is equal to the product of the transposes of the two vectors in 
reverse order. We can express this rule by 


(V.V3)' = WV. (8.4.4) 


From the rules for forming major product of two vectors, we can write 
Eq. (8.4.5). 


s bia bias vas AIR. 
WT m а... а= |00 be ... be, (8.4.5) 
bm [һа bma ... ba, 


Now suppose we write Eq. (8.4.5) compactly as 


AES (8.4.6) 
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From Eq. (8.4.5) you can see that the element in the jth row and ith 
column of the product is the jth element from the b vector and the ith 
element from the a vector. Therefore, we can write 


Fj; = Ьа; (8.4.7) 


But since the factors of a scalar product of two scalar quantities can be 
reversed, Eq. (8.4.7) can be written as 


Fj; = аф; (8.4.8) 


Now notice that the right side of Eq. (8.4.8) is exactly the same as the 
right side of Eq. (8.3.3). Therefore, the left sides of the two equations 
must also be equal, and we can write 

C; = Fy (8.4.9) 
But remember that the element in the ¿th row and jth column of a matrix 
15 the same as the element in the jth row and ith column of its transpose; 
therefore, because of Eq. (8.4.9) we can write 


C-F (8.4.10) 


Now let us go back to Eq. (8.3.2) and take the transpose of both sides of 
the equation to get 

(V,V) = C (8.4.11) 
Next we substitute the left side of Eq. (8.4.11) in the right side of Eq. 


(8.4.9) and get 
(VV = F (8.4.12) 


Then we substitute the left side of Eq. (8.4.6) in the right side of Eq. 


(8.4.12) to get 
(V, V)' = VV; (8.4.13) 


Now Eq. (8.4.13) is exactly the same as Eq. (8.4.4), which is what we set 
Cut to prove, Equation (8.4.4) states that the transpose of the major 
Product of two vectors is equal to the product of the transposes of the two 
Vectors in reverse order. This theorem is proved by Eq. (8.4.5) through 
(8.4.13), Actually, it would not be necessary to go through all of these 
Steps for you to сее that the statement is true. You could simply see by 
Mspection that the last matrix in Eq. (8.3.1) is the transpose of the last 
Matrix in Eq. (8.4.5). However, it is well to learn early how, by using 
Simple rules, step by step, we can prove very important and sometimes 
Complicated theorems with a minimum of mental effort and confusion. 


$5 The Product of a Vector and Its Transpose 


Of particular interest in the analysis of behavioral science data is the 
Product of a vector and its transpose. There are two such products, a 
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column vector premultiplied by its transpose and a column vector post- 
multiplied by its transpose. The first we call the minor product moment of 
a vector and the second the major product moment. 


The minor product moment 


The minor product moment of a vector is obviously a scalar quantity 
since it is a special case of the minor product of vectors. More specifically 


it is simply the sum of squares of the elements of the vector, as you can see 
from Fig. 8.5.1. 


1 


[L в 23 =1+3°+2=1+9-+4=14 
2 


Fig. 8.5.1 


In general symbolic form, this is Eq. (8.5.1). 
Vive = [n о... a| ® = а +0 +... + а (8.5.1) 


Since the minor product moment of a vector is a scalar quantity it is ob- 
viously equal to its transpose. 


The major product moment 


In getting the major product moment of a vector and its transpose, we 
postmultiply the column form of the vector by its row form, or conversely 
we premultiply the row form of the vector by its column form. We shall 
prove that the major product moment of a vector is a symmetric matrix. 
First, we write out the major product of the vector and its transpose as in 


Eq. (8.5.2). 
а a; ал05 ... аа, 
2 
У.У, =| ® [п а... a| 0 ... ал 
аһ аа аъ ... а 
(8.5.2) 
If we indicate the major product moment of the vector by C, we can write 
Eq. (8.5.3). 
dj аа ... аа, 
с-|®® a se Mic 


аһа) ал ... а, 
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'Then from Eqs. (8.5.2) and (8.5.3) we get 
KV. (8.5.4) 


Now from Eq. (8.5.2) we see that the ijth element is a;a; and the jith 
element is aja. We have, therefore, 


Ci = айп; (8.5.5) 
Cji = ао: (8.5.6) 


But since the factors on the right side of Eq. (8.5.6) are scalars, they can be 
reversed to give us 


Ci = айп; (8.5.7) 


You see now that the right sides of Eqs. (8.5.5) and (8.5.7) are exactly the 
Same; therefore, the left sides must also be the same, and we have 


C; = Cis (8.5.8) 


Now Eq. (8.5.8) says that the element in the ith row and jth column is the 
Same as the element in the jth row and ith column. This, you remember, is 
the definition of a symmetric matrix. Therefore, Eqs. (8.5.2) through 
(8.5.8) prove that the major product moment of a vector is a symmetric 
matrix, 

In the analysis of data from the human sciences we shall have many 
Occasions to use the major product moments of vectors. An example is 


Fig. 8.5.2. 
4 16 12 4 
HE 3 n-i 9 3 
1 4 3 1 


Fig. 8.5.2 


It is easy to see from а numerical example like this that the major product 
Must be a symmetric matrix; however, it is good practice for you to prove 
the theorem in simple steps for the general case, regardless of what partic- 
Чат numbers are used as the elements of the vector. The major product 
Inoment of a vector is used extensively in caleulating covariance matrices 
from data matrices. 


8.6 Undefined Products of Vectors 


We have described two different kinds of products of vectors — the 
Scalar, or minor, product and the major product. To get the minor product 
of two vectors, we postmultiply a row vector by a column vector. To get 
the major product of two vectors, we postmultiply a column vector by a 
TOW vector. To avoid confusion in the use of matrix algebra as a sort of 
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automatic problem-solving machine, it is well to know what possible vector 
products have no meaning within the scheme of matrix algebra. We shall, 
therefore, illustrate the two kinds of products which might be found but 
which are not meaningful for practical use. The first of these is the product 
of a column vector and a column vector. By neither of the two methods of 
vector multiplication can a column vector be multiplied by a column veetor. 
To emphasize the impossibility of such a thing, we give 


3 4 a bi 
2|X]|3]|- nonsense d» | X | b» | = nonsense 
4 ii аз, bs 

Fic. 8.6.1 


Figure 8.6.2 shows in compact notation that the product of a column vector 
by а column vector has no meaning. 


VaVe = nonsense 
Fia. 8.6.2 


Neither can a row vector be multiplied by а row vector to yield a useful 
product. The impossibility of such a. product is illustrated by 


{1 2 5] X[2 3 7] = nonsense [а а а] X [bi be bs] = nonsense 
Fic. 8.6.3 


The impossibility of the product of à row vector by a row vector is illus- 
trated in compact notation by 


ViVi = nonsense 
lic. 8.6.4 


You see, therefore, that in the product of two vectors, whether a minor or 
major product, one of the factors must always be a row and the other always 
acolumn. You cannot have two column factors or two row factors. Conse- 
quently, unless otherwise specified, one of the factors will always have a 
prime and the other will always be unprimed. This must be so because, 
unless otherwise specified, an unprimed vector is always a column vector 
and a primed vector is always a row vector. It has been mentioned that 
this practice is not consistently followed by all books or articles on matrix 
algebra; however, we shall follow the practice very closely. 


8.7 Vector Multiplication and the Commutative Law 


We have already seen that by the commutative law of addition we mean 


a +b = b + a, or, in numbers, 3 + 4 = 4 + 3. The commutative law of 


multiplication in arithmetic or scalar algebra means that ab = ba, or 
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similarly, 3 4 2 4 X 3. But the commutative law of multiplication 
does not hold for the product of two vectors. What you have taken for 
granted about numbers since early childhood is not true about the special 
kinds of matrices we call vectors. You can see very readily that vector 
multiplication is not commutative. Suppose we define two different vectors 
as in Fig. 8.7.1. 


1 4 

У. = |3 Y, = |2 
2 5 
Fig. 8.7.1 


Then the minor product of the two vectors is shown in Fig. 8.7.2. 
4 
ҮҮ = 1 3 2)/2/=4+6+4 10-20 
5 
Fic. 8.7.2 


The minor product of the two vectors is a scalar quantity or 20. But if we 
reverse the order of the factors, or as we say, if we commute the factors, 


we get Fig. 8.7.3. 


4 412 8 
Wvi-|2|ü 3 2 2|]2 6 4 
5 5 15 10 

Fig. 8.7.3 


That is, we get a major product. The two products shown in Figs. 8.7.2 and 
8.7.3 cannot possibly be equal because the first one is a scalar quantity and 
the second one is a matrix. 

Similarly, if we commute the factors in a major product of two vectors, 
We get a minor product, or scalar quantity. Furthermore, it is not possible 
to commute the factors in a major product unless the two vectors are of the 
Same order. Also, since commuting the factors in the scalar or minor 
product of two vectors gives the major product of two vectors, it follows 
that in so doing we always get a square matrix. This, of course, is true 
because in a minor product the two vectors must be of the same order, If 
the two vectors are of the same order, their major product must have the 
Same number of rows as columns and hence be a square matrix. 

We can show in compact notation that vector multiplication is non- 
commutative in Fig. 8.7.4. 

У.У ғ ViVa 
РУ; ғ УЬ 
Ета. 8.7.4 
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'The equals sign with a line drawn through it means that the expressions on 
the right and left are not equal. This symbol is sometimes read as "distinct 
from." Thus the first line of Fig. 8.7.4 would read, “VaV; is distinct from 
ViV.^ 


8.8 The Minor Product of Type 1 Supervectors 


Characteristics of the minor product 


From what we know about partitioned vectors or type I supervectors, 
it is fairly easy to deduce three simple rules about their minor products. 
First, in multiplication with type I supervectors, as with simple vectors, 
both factors must be of the same order to result in a minor product. Second, 
corresponding subvectors of the supervectors must also have the same 
order. Third, the minor product of two type I supervectors is a scalar 
quantity. These rules become obvious if we give the minor product of 
two type I supervectors in terms of their vector elements. As in multipli- 
cation of simple vectors, the prefactor in the minor product is a row vector 


and the postfactor is a column vector. Two supervectors are given in 
Fig. 8.8.1. 


Va Vn 
=|" | v, =| 
Ve, V, 

Fic. 8.8.1 


Then the minor product of these two supervectors can be illustrated by 
Eq. (8.8.1). 


Vo, 
Vim (Ve VL o2 Vad] | = VLY. + VV to VLYs 
Vo, (8.8.1) 


Here we have used the same rules for the minor product as we did in 
multiplying simple vectors; that is, we took the product of corresponding 
elements of the two vectors and added them together. You should notice 
in Eq. (8.8.1) that in the middle expression all of the subvectors of V/ are 
on the left of all of the subvectors of У». Similarly, in the last expression in 
Eq. (8.8.1), each V; subvector is on the left of each V» subvector. Since 
now the elements of the supervectors are themselves vectors, you are not 
free to interchange the order of the factors in the products of two elements 
as you were in multiplying simple vectors. This is because, as you have 
learned, vector multiplication is not commutative. 
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You will also see from the right side of Eq. (8.8.1) why corresponding 
vector elements in the two vectors must be of the same order. Each term 
in the right side of Eq. (8.8.1) is itself a minor product of two vectors. 
As we know, both factors in a minor product must have the same number of 
elements, Notice, of course, that there is no restriction on the number of 
elements in the subvectors within the supervector. That is, Va can have 
any number of elements, Va, any number of elements, and so on, as long as 
Va, has the same number of elements as Va, Va. the same number as V», 
and so on. To make Eq. (8.8.1) more meaningful in terms of actual num- 
bers, let us take the instance of two second-order type I supervectors whose 
Subvectors are as given in Fig. 8.8.2. 


nef -E 


1 2 
Vag =| 7 Va =] 1 


Fic. 8.8.2 


л 
o2 


We can now write the minor product of the two supervectors as in Fig. 
8.8.3. 


2 
1 à 9 
= 12 3 | 1 7 89|, |= 2 [|+ 7 о5]|1|= 
3 
1 
3 
х2+3хЦ+1х2+7х1+5хХ3]=7-+Е24 = 31 


Fie. 8.8.3 


The transpose of ће minor product 
We shall next prove that the transpose of the minor product of two type I 
Supervectors is equal to the product of their transposes in reverse order. 
What we shall prove is stated compactly as 
Vive = ViVa (8.8.2) 
We note first that the left side of Eq. (8.8.2) is expressed in terms of the 
vector elements of the two supervectors in Eq. (8.8.1). In the same way, 
We can express the right side of Eq. (8.8.2) in terms of the vector elements 


9f the two supervectors by 
Va 
УУ. = [р Vi se Vel Fs = Vp Vos + ViVa d... + Vio Von 


(8.8.3) 
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Now compare the right side of Eq. (8.8.1) with the right side of Eq. (8.8.3). 
Notice that each term on the right in Eq. (8.8.3) is a minor product of two 
vectors. Remember also that a minor product of two vectors is equal to the 
product of their transposes in reverse order. For the ith term on the right 
in Eq. (8.8.3), we could therefore write 


Vi Vs = У.Р, (8.8.4) 
Now because of Eq. (8.8.4), we can write Eq. (8.8.3) in the form of 
ViV, = VV, + ViVo +... + VL Vs, (8.8.5) 


Now notice that the right side of Eq. (8.8.5) is exactly the same as the 
right side of Eq. (8.8.1). "Therefore, the left sides of the two equations 
must be equal to each other, and we can write 


Vive = VeVi (8.8.6) 


But you see that Eq. (8.8.6) is exactly the same as Eq. (8.8.2), which is 
what we set about to prove. Again, as in some of our earlier proofs, you 
could argue from common sense or see intuitively that the minor product 
of two type I Supervectors would have to be the same as the minor product 
of their transposes in reverse order, without going through the steps from 
Eq. (8.8.1) through Eq. (8.8.6). It is well, however, to learn to use these 
Simple mechanical routine procedures to prove the obvious so that you will 
be able to use them to prove things which are not at all obvious. 

We said at the beginning of this section that the minor products of two 
type I supervectors exist if, and only if, they are of the same order and their 
corresponding vector elements are of the same order. Strictly speaking, 
this is true; however, you will have guessed already that if in a computation 
you have two type I Supervectors, you can reduce them to simple form 
to get a minor product or a scalar, even though the supervectors are not 
of the same order. You will recall that this principle holds in addition of 
Supermatrices in general. We do not violate or contradict the rule, however, 
by such an exception because when we reduce the supervectors to simple 
form we no longer have supervectors. 

An example of how the scalar product of type I supervectors is used in 
the analyses of data from the social sciences can be given. Suppose we have 
given a number of different tests to entering freshmen at a university. We 
enter these test scores on sheets or rosters. We use the basic principle 
described in Chapter 1 for a data matrix. Each column on the sheet 
represents a different test and each row on the sheet represents an entering 
freshman. But suppose we had many more entering freshmen than we 
could get on a single sheet, so that to enter all of the scores of all of the 
freshmen we have to have n sheets, Now, having entered the scores, we 
want to find the correlation between the two tests in the first two columns 
of the sheets. This means we must get the sum of cross products for these 
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two scores for all the people on n sheets. Actually, what we would probably 
do if we had a caleulating machine is to get the sum of cross products for the 
first sheet, then for the second, and so on, until we had a sum of cross 
products for each of the sheets. We would then add them all together to get 
the total sum of cross products for all of the entering freshmen on the two 
tests. To express in terms of matrix algebra what we are doing in this 
illustration, we let z in Fig. 8.8.4 represent a type I column supervector of 
the scores on the first test, and у represent a similar supervector of the scores 
on the second test. 


11 yi 
p= Vo y- к 
Tn Yn 

Fie. 8.8.4 


Each element in the supervector is a vector of the scores on a single sheet. 
Then Eq. (8.8.7) represents the sum of cross products of all of the sheets for 


the two tests. ) 
т'у = хул + xs +... а (8.8.7) 


Each term on the right side of Eq. (8.8.7) represents the sums of cross 
Products for a single sheet. 


8.9 The Major Product of Type І Supervectors 


The major product of type I supervectors differs in several important 
respects from the minor product. Here, as in multiplication for the major 
Product of simple vectors, the prefactor must be a column vector and the 
Postfactor a row vector. Here also the two vectors need not have the same 
order. Furthermore, there are no restrictions on the orders of the subvectors 
in either of the factors of a major product of type I supervectors. A fourth 
characteristic is that the major product is a supermatrix. The rule for 
forming the submatrices of the major product of two supervectors is the 
Same as that for forming the elements of the major product of simple 
vectors. The first row of matric elements of the product matrix is formed 
by multiplying each element of the postfactor by the first element in the 


Prefactor, , 
Equation (8.9.1) illustrates how we form the major product of type I 


Supervectors. 


Va Va Vi, Va Vi, кез Vai, 

VU. КЕК wu asl E 
Уу; = | Vs "I ph age Wee ү к T " ^ | У.У 
v, ViVi Жый 2.. FQ 


(8.9.1) 
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You see that in the right side of Eq. (8.9.1) each element in the matrix is 
the major product of subvectors. Therefore, the elements in this major 
product are themselves matrices. 

It should be clear from Eq. (8.9.1) that the supermatrix has the same 
number of rows as the prefactor and the same number of columns as the 
postfactor. Furthermore, it should also be clear that the height of a matrix 
element is determined by the order of the subvector from the prefactor and 
the width by the order of the subvector from the postfactor. Let us consider 


a numerical example of the major product of two type I supervectors. 
We let the subvectors be given as in Fig. 8.9.1. 


ч 
г 
| 
бк сл o t 


Fic. 8.9.1 


Then the major product will be as shown in Fig. 8.9.2. 


VaV; = [812|[2 12351] = 


H [3 12] | [2] A 2351] А 


2 2 2 z 
4 1812) 14 [2] || 4)[2 3 5 1] 1 
1 1 1 : 


Fig. 8.9.2 


єк PD | wd 


о њ 
о © 
- кє 
ao 
wr 


In this figure V, is a second-order column supervector, and V; is a third- 


order row supervector. The first element in V, is of order 2 and the second 
element of order 3; therefore, all submatrices in the first row of the product 
have two rows and all in the second row have three rows. 'The orders of the 
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three subvectors in the postfactor are respectively 3, 1, and 4; therefore, 
the number of columns in the first, second, and third matrix columns of the 
product is 3, 1, and 4, respectively. 


8.10 The Transpose of the Major Product of Two Type 1 Supervectors 


Again we shall prove by means of simple equations what may already 
be obvious to you. The transpose of the major product of two type I 
Supervectors is equal to the product of their transposes in reverse order. 
This rule is given as 

(V, Vi = Vo (8.10.1) 


First we will write out the elements of the supermatrix for the right side of 
Eq. (8.10.1) in Eq. (8.10.2). 


Vai, Va, ... Vall 
ру | s TAM, с» ТШ noy 
Yi, V?, Vins se Vs Vs. 


Here we have simply used the general rule for obtaining each element of the 
Product matrix in terms of the elements of the factors. Next, let us go back 
to Eq. (8.9.1) and take the transpose of both the right and left sides of this 
equation. Remember that to take the transpose of this supermatrix, we 
write rows as columns and take the transpose of each element. This we do 
ш Eq. (8.10.3). 


(И), (ы) у, (Ра, Vi 
Аы (8.10.3) 
(V,VLY С nee (Йй 


Now notice that each element of the supermatrix on the right side of Eq. 
(8.10.3) is the transpose of the major product of two vectors. Remember 
also that the transpose of the major product of two simple vectors is the 
Product of the transposes in reverse order. For the Zjth matrix element of 
Eq. (8.10.3), we have therefore 

(V,Vi) = ViV; (8.10.4) 
Substituting the right side of Eq. (8.10.4) for each of the elements on the 
Tight side of Eq. (8.10.3) we get Eq. (8.10.5). 

uS WU uc KAP 


E УЙ, us. FEL 
Ga =| afe atta hee (8.10.5) 


ae TUNE $us 
Vien Vases ccs Vie is 
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But now notice that the right side of Eq. (8.10.5) is exactly the same as 
the right side of Eq. (8.10.2). "Therefore, the left sides must also be the 
same and we can write 


(VaV) = РУ, (8.10.6) 
Now Eq. (8.10.6) is exactly the same as Eq. (8.10.1), which we started out 
to prove. 


The major product of type I supervectors plays an important role in 
the calculation of covariance matrices where attributes are partitioned into 
predictor and criterion sets. 


8.11 Special Vector Products 


Certain vector products, both minor and major, are of special interest 


because they occur so frequently. Some of the most common of these we 
shall now consider. 


The minor product moment 


The minor product of a vector and its transpose is the sum of squares of 
its elements. This we have already indicated in Section 8.5, but since the 
rule is so generally used in the analysis of behavioral science data it will be 
repeated here. Another example is given in Fig. 8.11.1. 

3 
[3 4 W)4)=34447=9416449 = 74 
С 


Fie. 8.11.1 


We repeat another form of the general rule in Eq. (8.11.1). 


X 


X 
а А SP Р 


X; 


In the summation notation commonl 
(8.11.1) can be written in the form of 


VxVx —[Xi Xs 


y used in elementary statisties, Eq. 


VxVx = ХХ? (8.11.2) 
Products involving the unit vector 


The minor product of the unit vector and a general vector is a scalar that 
is the sum of the elements of the general vector. It does not matter whether 


we use the unit vector as a prefactor or a postfactor. An illustration of this 
is Fig. 8.11.2. 
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3 1 
u 1 1]4]|=[3 4 7J|1|=3+4+7=14 
7 1 


Fic. 8.11.2 


Equation (8.11.3) shows this rule in algebraic form. 


Xi 
(rd ^ ен, 
X. 1 (8.11.3) 


To use the summation notation, we simply write Eq. (8.11.4) to show that 
the minor product of a vector and the unit vector is the sum of the elements 
in the vectors, 
l'Vx = Vel = BX (8.11.4) 
It is clear then that when you take a sum of experimental values you are 
getting the minor product of a vector of observations and the unit vector. 
The minor product moment of a unit vector is a scalar equal to the order 
of the vector. This is obvious from Fig. 8.11.3. 


па | а екл = 
1 
1 
Hard | = р+1і+1+1=4 
1 
Fic. 8.11.3 


Tf, as given in Section 3.15, we indicate the unit. vector by the number 1 
and if we use а subscript to indicate its order, this principle can be expressed 
algebraically as 

1, =n (8.11.5) 


In Eq. (8.11.5) the subscript n means that the unit vector has n elements, 
and Eq. (8. 11.5) simply says that when the unit row vector is postmultiplied 
by the unit column vector, we get a scalar quantity n that is equal to the 
Order of the vector. 

The minor product moment of a sign vector is also equal to its order. 
This can be easily verified since the elements of a sign vector are either 
+1 or —1, Their squares are therefore each 1. 

If we take the major product of the unit vector by another vector, we get 
а very simple type of matrix. If a row vector is premultiplied by a column 
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unit vector, the product is a matrix in which each row is equal to the row 
vector. Ап example of this is Fig. 8.11.4. 


1 а 4 7 

1118 4 7]5|3 4 7 

1 3.4 7 
Fig. 8.11.4 


One use of this product is in data analysis involving means. If from a 
matrix of measures we subtract from each measure of an attribute its mean 
for all the entities, we are actually subtracting from the data matrix the 
major product of a vector of means and a unit vector. 

The general rule is expressed in algebraic form by Eq. (8.11.6). 


Q4 ао ТЕ x аһ 
iac NS EE (8.11.6) 
а Qy e Gx 


The major product of a general column vector postmultiplied by a unit. 
vector obviously is a matrix in which all column vectors are equal to the 
prefactor. This is illustrated by Fig. 8.11.5. 


4 4 4 4 
3l a ass ж 8 
5 5 5 


Fie. 8.11.5 


In algebraic form this rule is given by Eq. (8.11.7). 


а а а 
па v. 9 а (8.11.7) 
аһ an ie е Gn 


The major product of two unit vectors is a matrix each of whose elements 
is 1, or unity. This is shown in Fig. 8.11.6. 


1 DRLE Е 

LE Pub S о a 

1 s ES, EG 
Fic. 8.11.6 


As in the general case, the order of the major product is determined by the 
orders of the two factors, the rows being determined by the prefactor and 
columns by the postfactor. 
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Products involving e; vectors 


The minor product moment of an e; vector is equal to 1. Remember that 
ап e; vector is a vector all of whose elements are 0 except one and that is 
unity. You can easily see why this rule holds, irrespective of the order of 
the vector, from Fig. 8.11.7. 


0 
(0 1 0 Of, f=oxo+1x1+0x0+0x0=1 
0 
Fia. 8.11.7 
The general rule is 
eie; = 1 (8.11.8) 


The minor product of any e; vector by some other e; vector is always 0. 
This is illustrated by Fig. 8.11.8. 
1 
(0 1 0 0 d —0х1+1х0+0х0+0х0=0 


0 
Fic. 8.11.8 


In algebraic form we can state the rule as 
ele; = 0 (8.11.9) 

Where т is different from 7 
The scalar product of an e; 
quantity that is the ith element of the vector. Thus 


eV = V'e; = Vi (8.11.10) 


vector and any other vector is a scalar 


A numerical example is given in Fig. 8.11.9. 
5 


4 
fo 1 0 05| =* 
2 


Fie. 8.11.9 


The minor product of any e; vector by & sdb vadis шу. Thus 


е1 = Пе = 1 (8.11.11) 


It is shown in numerical form in Fig. 8.11.10. 
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(9: 2270770] 


н фе HH 


Fie. 8.11.10 


The major product of an e; vector and an e; vector is an e;; matrix. You 
will recall that an e;; matrix is one all of whose elements are 0 except one, 
which is unity. Figure 8.11.11 shows the major product of an e; vector and 
ane; vector. You will notice in Fig. 8.11.11 that the position of the 1 in the 
prefactor determines its row in the matrix, whereas the position of the 1 in 
the postfactor determines the column position in the matrix. 


0 0 0 0 
1 0 = 


oor 
oor 


0 0 
0. 0 
0 0 
Fig. 8.11.11 


Notice also that, as in the general case of the major product of two vectors, 
there is no restriction on the orders of the two factors. Equation (8.11.12) 


is the algebraic form of the rule for the major product of an e; and an е; 
vector. 


ee, = е (8.11.12) 


Here the equation states that if ап e; column vector is postmultiplied by ап 
€; row vector, we get an e;; matrix whose unit element is in the ith row and 
jth column. We therefore have the obvious rule that any e 


| и matrix may be 
expressed as the major product of an €i vector by an ef vi 


ector. 


SUMMARY 


1. The minor product of two vectors: 

a. The left factor is a row vector and is called the prefactor. The right 
factor is a column vector and is called a postfactor. The column 
vector is said to be premultiplied by the row vector, and the row 
vector is said to be postmultiplied by the column vector. 


- The minor product is a scalar quantity that is the sum of products 
of corresponding elements of the two vectors: 


ViVe = [ш a ... a4] b | = aibi + adb: +... banda 
be 


bn 
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е. The minor product is sometimes called scalar product. 
d. The minor product of two vectors is defined only for vectors of 
equal order. 


- Transpose of minor product: 


a. It is equal to the minor product since it is a scalar. 
b. It is equal to the product of the transposes in reverse order, that is, 


VIV, = ViVa 
The major product of two vectors: 
a. A column vector is postmultiplied by a row vector, or a row vector 
is premultiplied by a column vector, that is, V, V; is a major product. 
b. In a major product of vectors having no zero elements the product 


is a matrix in which the zjth element is the product of the ith 
element of the prefactor and the jth element of the postfactor. 


а аһ аһ ... аф 

ab. Di oz. nb 

VeVi =| 8 [bi b; ... dn) = a 2 E T ч А 
а бару de 2. abs 


€. Rows are all proportional to one another, as are all columns. 

d. The order of the major product is the height of the prefactor and the 
width of the postfactor. For a major product of two vectors no 
restrictions are placed on the order of either vector. 


+ Transpose of major product: 


a. It is equal to the product of the transposes in reverse order. 
b. (УЙ)! = VIV, 
Product of a vector by its transpose: 


a. The minor product of a vector by its transpose we call the minor 
product moment of the vector. It is the sum of squares of the 


elements of the vector. : 
b. The major product of a vector by its transpose we call the major 


product moment. It is a symmetric matrix. 
Undefined products of vectors: 
a. A row vector cannot be multiplied by a row vector nor a column 


vector by a column vector. 
b. One of the factors must be a row and the other must be a column. 


* The commutative law: 


a. The factors in a major product of vectors can be commuted only if 
they are of the same order; the product is then a scalar rather than 
a matrix. 

b. The factors of a minor product of vectors can be commuted but the 
product is then a matrix rather than a scalar. 
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c. Vector multiplication is noncommutative, that is, 
Vive АГА 
even if VV; is defined. 


8. Minor products of type I supervectors: 

a. The minor product of two type I supervectors can be expressed if 
they are of the same super order and their corresponding subvectors 
are of the same order. 

b. The minor product of two type I supervectors is the sum of the 
minor products of corresponding subvectors. 


9. Major products of type I supervectors: 

а. No restrictions are placed on the superorders of either factor or on 
the orders of their subvectors. 

b. The ijth matric element of the product is the product of the ith 
column subvector of the prefactor postmultiplied by the jth row 
subvector of the postfactor. 

c. The product is a supermatrix in which the number of matric rows 
is the superorder of the prefactor and the number of matric columns 
is the superorder of the postfactor. 


10. Transpose of the major product of type I supervectors: It is equal to 
the product of the transposes in reverse order. 
11. Special vector products: 
a. The unit vector. 
(1) the minor product of a unit vector with a general vector is the 
sum of the elements in the latter. 
(2) The minor product moment of a unit vector is the order of the 
vector. 
(3) The major product of a unit vector postmultiplied by а general 
vector is a matrix all of whose rows are equal. 
(4) The major product of a unit vector premultiplied by а general 
vector is a matrix all of whose columns are equal. 


(5) The major product of two unit vectors is a matrix all of whose 
elements are 1. 


b. e; vectors. 
(1) The minor product: 
(a) With a unit vector, it is 1: 
е1 = l'e; = 1 
(b) With a general vector, it is the ith element of the vector: 
eV = V'e; = V, 
(c) With itself or minor product moment, it is 1: 


ее; = 1 
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(2) The major product: 


139 


(a) With general vector, if e; is the prefactor, it is a matrix all 
of whose rows are 0 except the ith, which is the vector; if 
e; is the postfactor, it is a matrix all of whose columns are 0 


except the ith, which is the vector. 


(b) With another e; vector, it is a matrix whose elements are 


all 0 except the ijth, which is 1. 


EXERCISES 


- (a) Express as the product moment of a vector 


4+ 36 + 25 = 65 


(b) What kind of product moment is this? 


1+3+4+2 = 10 


. Express as the product of a vector and unit vector 


5214 
a=|8 214 
3214 


- Express as the product of a vector and unit vector 


444 
6 6 6 
b= g 3 
38 8 


- Express the number 4 as the product of unit vectors. 


- Express as the product of unit vectors 


ien 

dot d ivi d 

(а) = 2414 y=], 11 
194 Ted 


- Express the number 1 as the product of e; vectors. 


Express in two ways the scalar 0 as the product of an e; and е» vector. 


+ Given the matrices 


0 00 
CO 150 000 

z2|0000 й=|1 00 
0000 000 
0000 


- Express as the product of a vector and a unit vector in two ways 
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10. 


115 


12. 


13. 


14. 


15. 
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(a) Express z as the product of e; vectors using numerical subscripts. 
(b) Do the same for y. 


(c) What are the orders respectively of the pre- and postfactors of z? 
(d) Of y? 


(a) Express in two ways the number 1 as the product of a unit vector and an 
ез vector 


(b) What kind of products are these? 


(a) Express as the product of vectors 


020 
[0-80 
ca ОКА 
040 
(b) What kind of product is it? 
(a) Express as the product of vectors 
000 
Ь=|0 00 
285 


(b) What kind of product is it? 
(a) Express as а product moment of vectors 


1-2 3 
a=|2 4 6 
3.69 
(b) What kind of product moment is it? 
Given e; is a third-order vector and 


as? dying Pa = 1 
What is V’? 


е, €», €; and e; are all fourth-order vectors 


(a) Express the sum of their major product moments by the appropriate symbol 
with subscript to indicate order 
(b) What is the sum of their minor product moments? 


ANSWERS 


2 
(a) [2 6 5]|6|— 65 (b) Minor product moment 
5 


1 
3 

[ой л) s fl 3 2 4 
4 


а EN 


6. 


1 
(a) z = То 
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(а) x = 1315 (b) y = 1415 


/ 
1 = е 


(b) у= ер (0) 54 (034 
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(b) Minor products (scalar products) 


(b) major 


(b) major 


(b) major 
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Chapter 9 


Matríx Multíplícatíon 


9.1 Matrices Expressed as Type Il Supervectors 


The a matrix as а type Il supervector 


Before considering the subject of matrix multiplication, we shall review 
the notation for matrices expressed as type II supervectors (Section 5.9). 
The rules for vector multiplication given in Chapter 8, can be applied to 
matrices in general if the matrices are regarded as supervectors. First, let us 


review how the matrix a may be expressed as a type II supervector in row 
form. 


@=[a1 аз ... a], (9.1.1) 


You will recall that the a.; elements on the right are the column vectors of 
the matrix a, that the internal subscript m means that there are m columns, 
and the external subscript n means there are n rows. You also recall that 
we can write a matrix as a type II column supervector as in Eq. (9.1.2). 


iu | 9 (9.1.2) 
а. |n 


Remember that the elements on the right of Eq. (9.1.2) are the row vectors 


of the matrix a, that the internal subscript n means that there are n rows 
and the external subscript m means there are m columns. 
The transpose of Eq. (9.1.1) is then Eq. (9.1.3). 


d 
a’ =| % (9.1.3) 
РА 
Тһе transpose of Eq. (9.1.2) is 
а = (а. а. ... a,], (9.1.4) 


MATRIX MULTIPLICATION 143 


The b matrix as a type Il supervector 


Suppose now that we have another matrix b with ¢ rows and s columns. 
We can write b in row supervector form as 


b = [ba bs ... Dale (9.1.5) 
We write it as a column supervector as 
bi. 
pa] ® (9.1.6) 
vd. 
The transpose of Eq. (9.1.5) is 
b 
p-.|^ (9.1.7) 
v, ]. 
The transpose of Eq. (9.1.6) is 
b = [bi be ... b (9.1.8) 


9.2 The Product of Two Matrices Expressed as the Minor Product of 
Type Il Supervectors 


The product of matrix a and matrix b can now be considered to be the 
minor product of type II supervectors. This means, of course, that we must 
have a row vector on the left and a column vector on the right. We there- 
fore use the right side of Eq. (9.1.1) for the a matrix and the right side of 
Eq. (9.1.6) for the b matrix. This gives us Eq. (9.2.1). 

bi. 
, 


b; 
ab = [a1 аз ... аһј| - (9.2.1) 


bi. Je 
To get the product of the vectors on the right side of Eq. (9.2.1), we use the 


Sàme rule as in multiplying simple vectors. We take the sum of the product 
of corresponding elements from the two vectors. This gives us 


ab = [a.sb}. + asdf. +... + ambt ]us (9.2.2) 


Now notice carefully three things about the order of the symbols in Eqs. 
(9.2.1) апа (9.2.2). First, on the left of both equations, a precedes b. The 
Prefactor is a and the postfactor is b. Second, on the right of Eq. (9.2.1) the 
Tow vector for a precedes the column vector for b; that is, the row vector 
for a is the prefactor and column vector for b is the postfactor. Third, on 
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the right in Eq. (9.2.2), a.; always precedes Ь;.. Once we start out with a 
given order, we must be very careful to preserve this order consistently. 
Keep this in mind until you learn certain exceptions to this rule and how to 
use them. 

Notice now that since each term on the right side of Eq. (9.2.2) is the 
product of two vectors, the first of which represents a column vector from a; 
the second, a row vector from b; this product, expanded, or in number form, 
is a matrix, and thus a major product. We see, therefore, that the product 
of two matrices may be expressed as a sum of major products of vectors. А 
numerical example of this rule should make the point clear. Let us repre- 
sent the two matrices a and b as in Fig. 9.2.1. 


(a.1) (a.3) 
2. 
е 1 2](bi) 
a-2|3 5 b= | | » 
6 1 3 110) 
Fic. 9.2.1 


In Fig. 9.2.2, we show the product, a X b, as the sum of two major products 


of vectors. In Fig. 9.2.3, we actually multiply out the product of the two 
pairs of vectors. 


(ал х bi) (as X b.) 


2 1 
ab —-|3| [1 2] 4-|5 fs 1] 
6 1 
Fic. 9.2.2 
2 4 $9 1 5 5 
ab-|3 6|-|15 5|-|18 11 
6 12 24 9 13 
Fic. 9.2.3 


The product of two matrices defined as the sum of major products of 
vectors is generally not given in textbooks on matrix algebra. However, 
this definition has been found very useful in the analysis of data from the 


behavioral sciences. Tt is particularly important in high-speed computer 
programming. 


9.3 The Product of Two Matrices 


Expressed as the Major Product of 
Type Il Supervectors 


А much more common way to express the product of two matrices than 
the one given above is as the major product of type II supervectors. This 
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means that the product a X b would be expressed as a column supervector 
postmultiplied by a row supervector. We therefore take the column super- 
vector form of a given in Eq. (9.1.2) and the row supervector form of b given 
in Eq. (9.1.5) and express the product as in Eq. (9.3.1) 

ai. 

alins Ба. bu ars ©; (9.3.1) 

Г In 

To get the product of the two supervectors on the right of Eq. (9.3.1), we 


use the same rule for getting the major product of simple vectors. This 
gives us 


a; ba aj b.» EE ai b. 
^ 
=з aba oes. аһ (9.3.2) 
aibi aba «os бый» 


Remember that each of the elements in the product matrix represents а 
TOW vector postmultiplied by a column vector. Therefore the right side of 
Eq. (9.3.2) is a matrix whose elements are the scalar or minor products of 
Vectors. Notice also that a comes before b on the left side of Eq. (9.3.2). So 
also the а, comes before b.; on the right side of Eq. (9.3.2). | 

We may take the same example given in Fig. 9.2.1 to show numerically 
how the product of two matrices may be expressed as a matrix whose ele- 
ments are the minor produets of vectors. Figure 9.3.1 gives the product 
Matrix before the calculation of minor products of the vectors. 


гаре 
ab =| [3 ails | (3 ДИ 
(6 Ш B (6 Ш H 


Fig. 9.3.1 


In Fig, 9.3.2, the multiplication for each of the elements is carried out. 


ILI PIXI 2x2--1X1 5- 5 
ab-|3x1-«5x3 3X2+5X1/=/18 11 
6xX1-1x3 6X2+1X%1 9 13 


Fic. 9.3.2 


You see that Fig. 9.3.2 gives exactly the same answer as Fig. 9.2.3, but we 
arrive at this result by two distinctly different computational procedures, 
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The minor product form for the product of two matrices is by far the most 
common among mathematicians. We shall, however, have occasion to use 
both the minor and the major products of type II supervectors for the 
product of matrices. 


9.4 Reversing the Order of the Factors 


The minor product form 


In the previous section, we considered the product of the matrices a and b 
where a was the prefactor and b the postfactor. Let us now consider the 
case where b is the prefactor and a the postfactor. First let us express this 
product in reverse order as the minor product of supervectors. For the b 
matrix, we use the right side of Eq. (9.1.5) and for the a matrix the right 
side of Eq. (9.1.2). This gives Eq. (9.4.1). 


ba = [bi be ... Б. 99 (9.4.1) 


a. ju 
If we multiply out the right side of Eq. (9.4.1), we get, 


ba = [baal + Баа +... + bun Jin (9.4.2) 
The major product form 


Next we express the product b X a as the major product of supervectors. 
We use the right side of Eq. (9.1.6) for the b matrix and the right side of Eq. 
(9.1.1) for the a matrix. This gives us Eq. (9.4.3). 

bi. 
b! 
Ше a эшш CE (9.4.3) 
bie 


If we multiply out the right side of Eq. (9.4.3) we get 


йал bias tas 
, , JAM 

ba = A бйз „.. б» (9.4.4) 
Мал Mas ... Dan 


9.5 Dimensions of Factors and Their Product 


We are now ready to draw 
orders of two matrices that are 
of their product. 


Some conclusions about restrictions on the 
to be multiplied together, and also the order 
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The common order 


If we look at the right side of Eq. (9.2.2), the minor product, we see that 
each column vector of a is paired with a row vector of b. Therefore, where a 
is the prefactor and b is the postfactor, in order for Eq. (9.2.2) to make any 
sense, we must have the same number of columns in a as we have rows in b. 
Then if a X b exists, Eq. (9.2.2) shows that m equals ¢. The same point can 
be made from the right side of Eq. (9.3.2), the major product. From Eq. 
(9.3.1) you can see that each of the row vectors from a has m elements in it, 
and each of the column vectors from b has ¢ elements in it. But in Eq. 
(9.3.2) each element is a row vector from a postmultiplied by a column 
vector from b. "Therefore the rows and columns must each have the same 
number of elements. Thus Eq. (9.3.2) also proves that m equals t if it is 
possible to postmultiply a by b. 

Now look at Eq. (9.4.2). Here the prefactor is b and the postfactor is a. 
In order for Eq. (9.4.2) to make any sense, the number of columns in b must 
be equal to the number of rows in a; therefore, in this case, s must equal n. 
The same rule can also be deduced from Eq. (9.4.4). Here each element isa 
TOW vector from b postmultiplied by a column vector from a, but Eq. (9.4.3) 
Shows that the row vectors from b have s elements and the column vectors 
from a each have n elements. If it is possible to postmultiply a row vector 
from b by a column vector from a, then they must both have the same num- 
ber of elements. Therefore s must be equal to n, if b times a exists. 

We are now ready to state the general rule that two matrices can be 
multiplied together if, and only if, the number of columns of the prefactor is 
equal to the number of rows of the postfactor. To put it another way, the 
Width of the prefactor must be the height of the postfactor. When these 
two dimensions are equal they may be called the common dimension or 
common order of the matrices. Therefore we have the rule that the common 
order of the two matrices of a product is the width (columns) of the left- 
hand matrix (prefactor) and the height (rows) of the right-hand matrix 
(postfactor). 

If two matrices do not have a common order, we say that they are non- 
conformable. If they have a common order, we say that they are conformable. 


The order of the product 


It is easy to see now what the order of a product is in terms of the orders 
of the factors. Equation (9.3.2) shows that the product ab has n rows and s 
Columns; or that it is of order n X s. Equation (9.4.4) shows that the 
Product ba has { rows and m columns or that it is of the order ¢ by m. But in 
Eq. (9.3.2), n is the number of rows in the prefactor and s is the number of 
columns in the postfactor. Similarly, in Eq. (9.4.4), is the number of rows 
in the prefactor and m is the number of columns in the postfactor. There- 
fore, the number of rows in the product of two matrices is the number of 
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rows in the prefactor, and the number of columns in the product is the num- 
ber of columns in the postfactor. These numbers we call the distinct dimen- 
sions or distinct orders of the factors. We therefore conclude that any two 
matrices may be multiplied together provided they are conformable, no 
matter how many rows the left-hand matrix has or how many columns the 
right-hand matrix has. 

We also see that the order of a product of two matrices does not indicate 
what the common dimension of the two factors is. All we know about the 
order of two factors from their product is their distinct orders and the fact. 
that they do have а common dimension. 


The rule for the dimensions of factors and the dimensions of products is 
summarized by Fig. 9.5.1. 


y 2 2 
н y b T c 
Fig. 9.5.1 


Here you see that the height of the matrix a is x; therefore, the height of the 
product cis z. The width of the matrix b is z; therefore, the width of the 
product cisz. The width of the matrix a is y; therefore, the height of the 
matrix b is y. 

Another way to remember the dimension rules for multiplication is to use 
the subseript notation to indicate the order of the matrices. Thus, we сап 
indicate the product of Fig. 9.5.1 by the equation in Fig. 9.5.2. 


абы = сы 


Fia. 9.5.2 


Here you see that the subscript y, which appears twice on the left, is а 
common dimension and disappears on the right side of the equation. Also, 
on the left, z is the first of the four Subscripts and on the right, x is the first 
of the two subscripts, whereas, on the left, z is the last of the fone subscripts, 
and on the right, z is the last of the two subscripts. When subscripts are 
used to indicate the order of the matrices, the last subscript of the prefactor 
must always be the same as the first subscript of the postfactor. Also, the 
first subscript of the prefactor must always be the first subscript of the 


product, and the last subscript of the postfactor must always be the last 
subscript of the product. 
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9.6 The Commutative Law of Matrix Multiplication 


You have seen that the commutative law for the addition of matrices 
holds. You also know that the commutative law for the multiplication of 
scalar quantities and numbers hold. Now let us consider matrix multiplica- 
tion. Does the commutative law still hold? In other words, is a X b equal 
to b х а? You have seen this is not true for vector multiplication. To 
answer the question, let us compare the ijth element of ab with the ijth 
element of ba. From Eq. (9.3.2), we see that the ¿jth element of ab is a; b.;. 
From Eq. (9.4.4), we see that the ijth element of ba is b; а; If we indicate 
the product ab by c, 


ab=c (9.6.1) 
and the product ba by f, 
ba =f (9.6.2) 
then, the ijth element of c in Eq. (9.6.1) is given by 
су = aibi (9.6.8) 
Similarly, the ijth element of f in (9.6.2) is given by 
fü = bia; (9.6.4) 


Now the right side of Eq. (9.6.2) is the scalar product of the ith row of a and 
the jth column of b, whereas the right side of (9.6.4) is the scalar product of 
the ith row of b and the jth column of a. These two would not in general be 
equal, since they consist of entirely different elements. Therefore, the right 
sides of Eqs. (9.6.3) and (9.6.4) are not in general equal, and consequently 
the left sides would not in general be equal. We can, therefore, write Eq. 
(9.6.5) to show that the two are not equal. 

eu zr fij (9.6.5) 
he matrices c and f 


Equation (9.6.5) shows that corresponding elements of t 
i ore, c and f are not 


in Eqs. (9.6.1) and (9.6.2) are not in general equal; theref 
1n general equal, and we can write the inequality 


T, (9.6.0) 


Substituting the left side of Eqs. (9.6.1) and (9.6.2) into the left and right 
Sides respectively of Eq. (9.6.6), we have the inequality 


ab з ba (9.6.7) 


We must conclude, therefore, that the commutative law for multiplication 
does not hold, in general, for matrix algebra as it does for scalar algebra. In 
other words, we say that matrix multiplication is noncommutative. There are 
Certain special cases when the commutative law of multiplication does hold 
In matrix algebra, some of which we shall consider later. 
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A numerical example confirms that matrix multiplication could not in 
general be commutative. We may let the matrices a and b be given as in 


Fig. 9.6.1. 
3. 2 2 4 
eb 1 ‚-[: 1 


Fig. 9.6.1 


"Then the product ab is given by Fig. 9.6.2. 


a = |} A 2 4| [12 14 
7 SNUB qu 99: 98 
Fic. 9.6.2 


On the other hand, the product ba is given by 


-[2 4][3 2] [34 24 
м = |2 1 E а "d 


Fic. 9.6.3 


The right sides of Figs. 9.6.2 and 9.6.3 are obviously not equal, and if you 
actually multiply out the appropriate rows from the prefactor by the ap- 


propriate columns from the postfactor, you can see why they could not be 
equal. 


9.7 The Associative Law of Multiplication 


We saw in Sections 7.2 and 7.3 how the associative law holds for the addi- 
tion and subtraction of matrices, Now let us consider the law as applied to 
multiplication of more than two matrices. First, note that in arithmetic and 
scalar algebra, of course, any number of quantities may be multiplied. In 
matrix algebra, any number of matrices may be multiplied, provided adja- 
cent pairs of factors have a common order. 

Therefore, as you might suppose, the number of rows in the first, or left- 
hand, factor of a series of factors determines the number of rows in the 
product. The number of columns in the last, or right-hand, factor of the 
series determines the number of columns in the product. The orders of the 
factors between the first and last factor in the series have nothing whatever 
to do with the order of the product. 
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Figure 9.7.1 illustrates diagrammatically how the order of the product is 
determined only by the first and last factors and how adjacent pairs of 
factors must be conformable. 


b c d e e 
Fra. 9.7.1 


In Fig. 9.7.1, you sce that the width of a preceding matrix is the same as the 
height of the matrix immediately following. For example, the second 
matrix v has a width of c and the third matrix w has a height of с. You also 
see that the first matrix u in the series has a height of a and the product 
matrix y has a height of a. Similarly, the last matrix x has a width of e and 
the product matrix y has a width of e. 

We can illustrate the same rule for order or dimensionality by using the 
Subscript notation to indicate the order of the matrices as in 


UaWocWealde = Yae 


Fic. 9.7.2 


Here you see that the last subscript of a matrix is the same as the first sub- 
Script of the following matrix. You also see that the first subscript of the 
first matrix in the series is the same as the first subscript of the product 
Matrix, and the last subscript of the last matrix in the series is the same as 
the last subscript of the product matrix. 
Let us take a numerical example, Fig. 9.7. 
Product of more than two matrices. 


t dt JE J- 


a с 


3, to show how we can get the 


Fia. 9.7.3 


Now there are two ways that we can get the product here. First, let us get 
the product ab and call this r so that we can have Fig. 9.7.4. 


a---[ IE -[ 8 


Fia. 9.7.4 


152 SIMPLE MATRIX COMPUTATIONS 


Then let us multiply the matrix т in Fig. 9.7.4 by the postfactor c. This 
gives us the product of all three matrices, as shown in Fig. 9.7.5. 


-[9 1][s 2] _ [59 60) _ | 
"Tls sila s3]|-]|àa |=? 
Fic. 9.7.5 


We can obtain this triple product a second way. First, we can get the 
product bc, which we may call s as in Fig. 9.7.6. 


mae fe 418 21 fiu £5 
Sr bee Ty piles la m 
Fic. 9.7.6 


Then we can premultiply s in Fig. 9.7.6 by the matrix a to get the triple 
product, as shown in Fig. 9.7.7. 


d 2 3//19 18] _ [59 60 P 
OLDI sei gas ga) = 
Fic. 9.7.7 


Now you see that Figs. 9.7.5 and 9.7.7 give exactly the same answer. It 
does not matter whether we first premultiply b by a and then postmultiply 
the product by c, or whether we first premultiply c by b and then pre- 
multiply the products by a. So long as we do not change the order of the 
matrices in the series, the product is the same. We can state this principle 
in algebraic form for the case of three factors as 


(ab)c = a(be) (9.7.1) 


If we have four factors in a product, the various combinations shown in 
Fig. 9.7.8 can be used. 


(1) abd =abed (4) abd =abed 
[2 


(2) abcd 


Ш 


(3) abcd = 


Fig. 9.7.8 
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The first equation in Fig. 9.7.8 means that first the product ab is formed, 
second the product cd is formed, and third, the product of these two 
products is formed. The second equation in Fig. 9.7.8 means that first 
Ње product ab is formed, second the product of this product with c is 
formed, and third, the product of this triple product with d is formed. 
The other equations in Fig. 9.7.8 are interpreted in the same way. You see 
then that it does not matter how we form the multiple product for more 
than two factors, as long as we continue to multiply adjacent pairs of 
matrices and do not change the order of any matrix in the original series. 
"Therefore, the rule is clear that matrix multiplication is associative for 


any number of factors in the product. 


9.8 The Order of Multiplication for Multiple Products 


Since matrix multiplication is associative for any number of factors, we 
may get the product by any combination method, so long as we do not 
change the order of the factors. Nevertheless, one method may be prefer- 
able because it requires least time and work. 

To determine the most economical order of multiplication, we first need 
a rule to determine the amount of work required in getting the product of 
any two matrices. It is easy to show that the number of scalar multiplica- 
tions involved in multiplying two matrices together is the triple product of 
their common dimension and their distinct dimensions. For example, let us 
indicate the orders of the matrices by subscript notation and let the product 
of two matrices a and b be given as c as in Fig. 9.8.1. 


ауу = Cz 


Fia. 9.8.1 


The matrix a has x rows; therefore, the matrix c has x rows. The matrix b 
has z columns; therefore, the matrix c has z columns. The total number of 
elements in the product c is zz. But to get each element of c requires the 
multiplication of y different pairs of scalar quantities since y is the common 
dimension of a and b. Therefore, the total number of multiplications 
required to get the elements in c is xyz. But this, as we have seen, is the 
triple product of the distinct dimensions and the common dimensions of 
the factors. This principle is illustrated in Fig. 9.8.2. 

3 2 : 


№ 


а b е 
Fia. 9.8.2 
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Here we see that a is a 4 by 3 matrix and b is a 3 by 2 matrix; therefore, 
the product c isa 4 X 2 matrix. But since the common dimension of a and b 
is 3, each element in the product matrix c requires the multiplication of 
three pairs of elements. Since there are 4 X 2 of these elements, the total 
number of scalar multiplications is 4 X 3 X 2 or 24 scalar multiplications. 

In determining the most economical sequence of multiplications for a 
series of factors we will not go far wrong if we consider only the total 
number of scalar multiplications for any particular sequence. It is true 
that scalar additions are also involved but these take much less time than 
do the scalar multiplications. This is true whether desk calculators or 
high-speed electronic computers are used. 


9.9 The Associative Law and Algebraic Checks 


Numerical computations may be checked by two general methods. You 
may simply do the work over again to see if you get the same results, or 
you may use algebraic checks. The associative law can be applied to the 
use of algebraic checks in computational work. Let us first take a very 
simple example of what is commonly known as cross footing. Suppose we 
have a simple table of numbers as in F ig. 9.9.1. 


4 


15 14 


Fic. 9.9.1 


Here the last or fourth column on the right of the table is the sum of the 
numbers in the rows, and the bottom row is the sum for each column. 
Notice that the sum for the fourth column is 43 and the sum for the last 
row is also 43. This number 43 therefore checks both the row sums and 
the column sums. Let us see now how the associative law of multiplication 
applies to a check of this sort. Suppose we let x be the original table of 
numbers and we postmultiply it by a column unit vector as in Fig. 9.9.2. 


3.45 1 12 

_ 12 1 8 9 
zl = 432 1| = 9 
5 v t 13 
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Next let us premultiply т by a row unit vector, as in Fig. 9.9.3. 


= [14 15 14] 


єк о с л 


Fic. 9.9.3 


You see that Fig. 9.9.2 gives us a column vector of row sums and Fig. 9.9.3 
gives a row vector of column sums. 

Now let us premultiply the column vector in Fig. 9.9.2 by a row unit 
vector as in Fig. 9.9.4. 


12 
Vil) 2( 1 1 1l] 9 = 43 
13 
Fic. 9.9.4 


Let us also postmultiply the row vector in Fig. 9.9.3 by a column unit 
vector as in Fig. 9.9.5. 


l 
(/z = [14 15 14] H = 43 
1 


Fia. 9.9.5 


You see that Figs. 9.9.4 and 9.9.5 give the same results. We therefore 
can write in matrix notation 


l'(x1) = (I'2)1 (9.9.1) 


Obviously, then, both the sum of the totals column and the sum of the 
totals row is a triple product of the row unit vector, the matrix, and the 
column unit vector in that order. Since the associative law holds for matrix 
multiplication, it does not matter whether we multiply the last two factors 
together first and then premultiply by the first. factor or whether we 
multiply the first two factors and then postmultiply by the last factor. 
This is one of the simplest examples of the application of the associative law 
to the checking of computations. 

Next let us consider a somewhat more complex application of the associa- 
tive law in checking matrix multiplication. We take as an example the 
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matrices a and b given in Fig. 9.2.1. The product of these matrices is given 
in Fig. 9.9.6. 


9 S 
ab —|18 11 
9 13 

Fig. 9.9.6 


Let us now postmultiply Fig. 9.9.6 by a column unit vector, as in Fig. 9.9.7, 
D. 35 1 10 
(a) 2|18 11 H = | 29 
9 13 22 
Fic. 9.9.7 


and next postmultiply the matrix b by a column unit vector, as in Fig. 9.9.8. 


"=k J-E] 


Then we may postmultiply the matrix a b 


y the vector obtained in Fig. 9.9.8, 
as in Fig. 9.9.9. 


2.1 3 10 
abl) —|3 5 || =| 29 
& 1 22 


Fic. 9.9.9 


If now you compare the vectors on the right side in Figs. 9.9.7 and 9.9.9, 
you will see that they are exactly the same. F 
know, of course, they must be the same becau 
same as indicated in 


rom the associative law you 
se the left sides must be the 


(ab)1 = a(b1) (9.9.2) 


Equation (9.9.2), together with Figs. 9.9.6 through 9.9. 
the associative law of multiplication to check the multiplication of the 
product, ab. After getting the product of the two matrices, we sum the 
rows of the product to get а column vector as on the right of Fig. 9.9.7. 
'Then we postmultiply the matrix a by a vector of row sums of b as in 
Fig. 9.9.9. The two resulting vectors must be the same if our computations 
have been correct. 


Another check on the product of the two matrices is 


9, shows how we use 


l'(ab) = (1'a)b (9.9.3) 
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Let us interpret this equation in terms of actual computations. According 
to the left side of Eq. (9.9.3), we take the column sums of the product 
matrix, ab. This is illustrated by Fig. 9.9.10. 


5 65 
(ab) = П 1 1]|18 1 | = [82 29] 
9 13 


Fic. 9.9.10 


Next we get the row vector of column sums for the a matrix as in Fig. 9.9.11. 


2 1 
la-[ 1 1/3 5 |= 1 7) 
бл 


Fig. 9.9.11 


Finally, we postmultiply the row vector of column sums of a in Fig. 9.9.11 
by the matrix b as in 


fpe 1210.27 
(l'a) = [11 7] E 2| = 2 29] 
Fic. 9.9.12 


Figure 9.9.12 gives the right side of Eq. (9.0.3). If you compare the right 
Sides of Figs. 9.9.10 and 9.9.12 you will see that they are exactly the same, 
às they should be according to the associative law expressed in Eq. (9.9.3). 


9.10 The Distributive Law and Matrix Multiplication 


If we take the sum of two or more numbers and then multiply this sum 
by another number, we get the same results as if we had first multiplied 
each of the numbers in the sum by this multiplier and then taken the sum 
of the products. This principle is illustrated by Fig. 9.10.1. 

23 + 4 + 5) = 6 +8 + 10 = 24 
2(3 + 4 + 5) = 2 X 12 = 24 
Fig. 9.10.1 
This principle is known as the distributive law. The distributive law of 


scalar algebra or arithmetic also applies to matrix algebra. We can state 
the distributive law of matrix algebra as 


alb + c) = ab + ac (9.10.1) 


Equation (9.10.1) can be generalized to include the sum of more than two 
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matrices multiplied by another. This is shown in Eq. (9.10.2), where the 
subscripts inside the parentheses refer to different matrices. 

a(bi + bz +... + B = ab; + а, +... + ab, (9.10.2) 


Assuming that the orders of the matrices are conformable, we could just 
as well have written the matrix a аза postfactor as in 


(bi + bo +... + ,)а = bia + Ба +... + bra (9.10.3) 


You notice that on the right of Eq. (9.10.2), the matrix a precedes each of 
the b matrices while in Eq. (9.10.3), the matrix a isa postfactor for each of 
the b matrices. The reason for this you will recall is that matrix multipli- 
cation is not in general commutative. 


Аз а numerical example of the distributive law for matrix algebra, in 
Fig. 9.10.2 we have three matrices, a, b, and c. 


ael 1-4 d а 
2 [4-2 eoa 
Fic. 9.10.2 
Substituting from Fig. 9.10.2 into the left Side of Eq. (9.10.1) we have 


Fig. 9.10.3. 
| alle 2| [19 8 
2 1116 3|" [50 7 
b+c 


a 


Fre. 9.10.3 


Substituting from Fig. 9.10.2 into the right side of Eq. (9.10.1), we have 


Fig. 9.10.4. 
Е IE 8 3| [19 8 
10 4 10 3| [90 7 
ab ac 
Fig. 9.10.4 


We see that the right sides of Figs. 9.10.3 and 9.10.4 are exactly the same, 


preferred to the method illustrated in Fi 
are required. That is, ordinarily, we get the sums of the matrices first and 
then multiply them by the common multiplying matrix, because it is much 
easier to add matrices than it is to multiply them. In Fig. 9.10.3 we have 
only one matrix product to compute, whereas in Fig. 9.10.4 we have two. 
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The rule does not always hold, however. It is possible in special cases that 
computational economy would result if the multiplieations were carried out 
before the additions. 

Thus far, Eqs. (9.10.1), (9.10.2), and (9.10.3) have shown how the dis- 
tributive law works in matrix algebra if we have the sum of matrices either 
pre- or postmultiplied by another matrix. We can also have a sum of 
matrices premultiplied by one matrix and postmultiplied by another. Here, 
too, the distributive law would hold. This situation would be written as 


a(b, +b: +... + bn) = abic + abc +... Б а.с (9.10.4) 


Notice in Eq. (9.10.4) that because in matrix algebra multiplication is non- 
commutative, on the right side of the equation the b matrix is always in the 
middle while the a matrix is on the left and the c matrix on the right. The 
order of these matrices must never be changed except in special cases which 
we shall consider later on. 


9.11 Algebraic Checks and the Distributive Law 
We have seen how the associative law of algebra can be used for checking 


computations. The distributive law can also be used for checking certain 
kinds of computations. Suppose we wish to add the matrices a and b given 


in Fig. 9.11.1. 
32 4 i 
as|i7| ъ=|1 3 
E 2 5 


Fie. 9.11.1 
'Тһе sum of the two matrices is given in Fig. 9.11.2. 
3 2 4 1 7 8 
а+Ь=|1 7|+|1 3]/=]2 10 
4 1 2 5 6 € 
Fig. 9.11.2 


Now let us postmultiply both a and b by a unit vector to get the row sums 
for each of the matrices, as in Fig. 9.11.3. 


3 2|n 5 4 1 1 5 
al-|i 7 [| =/8 bl=|1 3 [i] =|4 
4 I 5 2 5 7 


Fig. 9.11.3 
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Next let us postmultiply the right side of Fig. 9.11.2 by the unit vector to 
get a column vector of row sums. This is given in Fig. 9.11.4. 


т з 10 
(a+b)1=]2 10 ВЕ 12 
ME 12 


Fig. 9.11.4 


We also add together the right sides of both equations in Fig. 9.11.3 to get 
Fig. 9.11.5. 


5 5 10 
al+bl=/8]+]4] =] 12 
5 7 12 


Fia. 9.11.5 


You see that the right sides of Figs. 9.11.4 and 9.11.5 are exactly the same. 
Their left sides are also the same, then, and we can write 


(a + b)! = al 4- b1 (9.11.1) 


But Eq. (9.11.1) must be true 
distributive law where the su 
the unit vector. 


, for in general it is simply a special case of 2. 
m of two matrices has been postmultiplied y 
9.12 The Transpose of a Product of Matrices 

The transpose of a product of two matrices 


Let us now consider the transpose of a product of two matrices. If we 8° 
back to Eq. (9.3.2) and take the transpose of both sides, we get Eq. (9.12.1). 


a; b. a; b.» ves Da | 
(ab)! a a5 ba az b.s ©#% а, (9.12.1) 
айл аз .. \ uL. 


But we know that to get the transpose of a matrix we write rows as columns 
and take the transpose of each elemen 


t. Therefore, we can rewrite Ed. 
(9.12.1) as Eq. (9.12.2). 
b.m. Ыза». ba, 
(aby = | Um. Pom. ... bhan, (9.12.2) 
Mum. Ua. Бла 
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Let us next see what happens if we use Eqs. (9.1.8) and (9.1.4) to write 
Eq. (9.12.3). 


b^ 
b's 

ba =| ^? | [a о. . бав (9.12.3) 
U. t 

We multiply out the right sides of Eq. (9.12.3) to get Eq. (9.12.4). 

bua Dada: sos bas. 

bla! = bau. bo. ... bioan. (9.12.4) 
b',a. bids es bisan. 


But now compare the right sides of Eqs. (9.12.2) and (9.12.4), and you will 
see that corresponding elements are identically the same. Therefore, the 
left-hand sides of the equations must be equal, and we write 


(ab)! = Ба (9.12.5) 
We can therefore state the rule that the transpose of a product of two 
matrices is equal to the product of their transposes in reverse order. You 
recall that this is the same as the rule for the product of vectors. A numeri- 
cal example of the rule is given in Figs. 9.12.1 and 9.12.2. 


‚(з 2][2 4]Y _ |12 ЧЕ A 
ax- (G (1) 08 =la m 
a b 
Fic. 9.12.1 


ыле (27 A 
v a LENA 9 14 33 
; 


Fig. 9.12.2 


tiplied the matrix a by b and then 
in Fig. 9.12.2 we first took the 
fa. As you see, both 


Notice that in Fig. 9.12.1 we first postmul 
took the transpose of the product, whereas, 
transpose of b and postmultiplied it by the transpose o: 
methods give the same result. 

The transpose of a product of any number of matrices 
Let us see now whether, knowing the rule for the transpose of the product 
Of two matrices, we can deduce the rule for the transpose of the product of 
any number of matrices. If we indicate the product ab by k, 


ab =k (9.12.6) 
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and the product kc by f, 
ke =f (9.12.7) 


from the rule for two matrices, we can indicate the transpose of f in Eq. 
(9.12.7) as 


Р = (kc) = ck’ (9.12.8) 
From Kq. (9.12.6), we get the transpose of k as 
k' = (ab)! = Ба (9.12.9) 


If we substitute the right side of Eq. (9.12.9) for in the right side of Eq. 
(9.12.8), we can write 

f = c'b'a! (9.12.10) 
But if we substitute the left side of Eq. (9.12.6) in the left side of Eq. 
(9.12.7), we have 


abc =f or f= abc (9.12.11) 


Equations (9.12.10) and (9.12.11) show that the transpose of the product of 
three matrices is equal to the product of their transposes in reverse order. 


By this same line of reasoning, if we have a matrix a that is the product of 
any number of matrices аза, to an, as in 


а = @0...а, (9.12.12) 


then by the same methods we have 


just used, we can express the product of 
a’ as 


F 


Е, a аза! (9.12.13) 


We therefore have the very important rule that the transpose of the product 


of any number of matrices is equal to the product of their transposes in re 
verse order. 


SUMMARY 


1. Matrices expressed as type II supervectors: 
а. The matrix a 


a, 
+ 
а= [ai а» as], = | 9 
ал. Дь 
а 
, 
а. 
а =[a. а. Gs. lx 
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b. The matrix b 
b; 
b = [61 be ... bh а 
у, 1, 
2. The product ab as the minor product of type II supervectors: 
a. The equation in minor product form 
b. 


V. qoas ай E ullo 


ab = [01 аз ... Ql 

bi. Je 
b. The product of two matrices expressed as the minor product of type 
II supervectors is a sum of major products of simple vectors—the 


first factor of a given term coming from the corresponding column 
vector of the prefactor and the second from the corresponding row of 


the postfactor. 


3. The product ab as the major product of type II vectors: 
a. The equation 


aj. aibi abs жам alb. 

, + £ , 
giu ls he os dea] t M Же 

ai dus albw abs x08. 0a 


b. The product of two matrices expressed as the major product of type 
II vectors is a matrix of minor products of vectors where the ijth ele- 
ment is the minor product of the ith row of the prefactor postmulti- 
plied by the jth column of the postfactor; that is, if 


ab = с саз = а. 


4. Reversing the order of the factors: 
a. The minor product 


а. 
ba = (b ba ... Dade} "| = baas, + Бааз +... + Бла, Jim 


b. The major product 


, A , 
bi. иал bias ... Шаһ 


t 3 , 
bs. bats аз ... Ыйы 


ba = [йз 4. .. баја = 


, ud What ai Сш, 
bi. Бал Баз ... bla 
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5. Dimensions of factors and their products: 
a. Restrictions on the orders of the factors. 
(1) If ab exists, m and t must be equal; that is, the number of 
columns of a and rows of b must be the same 
(2) If ba exists, s and n must be equal; that is, the number of 
columns of b and rows of a must be the same 
b. Order of the product—The order of a product is the height of the 
prefactor and the width of the postfactor. 
c. Terminology for dimensions of the factors. 
(1) The height of the prefactor and the width of the postfactor are 
called the distinct orders or dimensions of the factors 
(2) The width of the prefactor and height of the postfactor are 
called the common order or common dimension of the factors 


d. The rules of order for matrix multiplication may be indicated with 
Subscript notation by 


ау: = C. 


6. The commutative law and matrix multiplication: 
а. ab and ba exist only if the orders of q and b’ are the same. 
b. Even if ab and ba exist, they are not in general equal; therefore, the 
commutative law does not in general hold for matrix multiplication. 


7. The transpose of a product of two matrices is equal to the product of 
their transposes in reverse order, that is, 


(ab)! = b'a’ 
8. The associative law of matrix multiplication: 


a. The product of more than two matrices may be formed by finding 
successively the products of any two adjacent factors, for example, 


(ab)c = a(bc) 


b. The order restrictions on more than two factors and their product 
may be indicated in subscript notation as follows: 


UaV oW aX ae = Y, 


9. The associative law and al 
two matrices may be chec 


ae 


gebraie checks: The computed product of 
ked by the associative law as follows: 


la (b b1) = [ab a(b1)] (ab)1 = a(b1) 
or alternately 


[+ Ка E l'[ab] = [l'a] 
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10. The distributive law of matrix multiplication is given by 
albi + bs +... + bn) = ab, + ab: +... + abn 

or 
(by + be +... + baja = bia + ba +... + bna 


11. The distributive law may be used in checking the addition of matrices 
as follows: 


[a al]d-[b b1] = (6 + b), (al + 61)) 
al-4- bi = (а +b)! 


a BNI T a+b 
HA id А 2 [e + A^ 


l'a 4- 1% = V[a + b] 
mber of matrices is the product: 


or alternately 


12. The transpose of the product of any nu 
of their transposes in reverse order, that is, 


(а...) = d»... 2h 


EXERCISES 


Assume that the following products all exist: (a) zy (uu (0 ту 


(d) ya’ 
+ If these products have all been expressed 
Vectors, what is the klth element for the transpose 


2. If these produets have all been expressed as the minor product of type II super- 
Vectors what is the kth term in the sum of vector products for the transpose of 


each of the products? 


as the major product of type II super- 
of each of the products? 


ANSWERS 
1. (а) Vai, (b) zu; (c) укта, (d) 21.01. 
2. (a) ya^, (улл уь (0) vento (d) ze 


Chapter 10 


Specíal Matríx Products 


10.1 The Multiplication of a Matrix by Its Transpose 


In the analysis of behavioral science data, one of the most common types 
of products is that of a matrix and its transpose. The matrix is usually а 
data matrix. As you know, there may be two kinds of products of a matrix 
and its transpose. The matrix can be premultiplied by its transpose or it 
can be postmultiplied by its transpose. The product of a matrix by its 
transpose is called the product moment matrix of the original matrix. 
Clearly, since the natural order of the matrix is vertical, the order of the 
product moment matrix is smaller when the matrix is premultiplied by its 
transpose than when the matrix is postmultiplied by its transpose. We 
shall therefore refer to the product of a matrix premultiplied by its trans- 
pose as the minor product moment of the matrix. The product of a matrix 
postmultiplied by its transpose will be referred to as the major product 


moment of the matrix. This use of the terms minor and major follows their 
use as applied to the products of vectors. 


The minor product moment 


Let us now prove that the minor product moment of a matrix is à 
symmetric matrix. You recall that we can represent the matrix а as а 
type II row supervector, thus: 


а= [01 аз ... a,], (10.1.1) 


Then the transpose of a can be written from Eq. (10.1.1) as Eq. (10.1.2). 


di 
а = |9 (10.1.2) 
а, |, 
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From Eqs. (10.1.1) and (10.1.2), the minor product moment matrix of a 
can be written as Eq. (10.1.3). 


aa - |^? | [ал а» ... ан) (10.1.3) 


In multiplying out the right side of Eq. (10.1.3), we have Eq. (10.1.4). 


, + , 
азал Gia. ... Alm 
СА + , 
Q24. a2 ... G2, 
аа = | "ес (10.1.4) 
аһа d ma ... тат 


Now notice that the ijth element on the right of Eq. (10.1.4) is а'а.;. 
Notice that the corresponding jith element is a’ ;a.;. But these two elements 
are equal since they are minor products of vectors and orie is the transpose 
of the other; therefore we can write 


а'а.; = a’ ач (10.1.5) 


But a symmetric matrix is one whose ijth element is equal to its jith 
element. Therefore, the minor product moment matrix of a is symmetric. 

A numerical example of a minor product moment matrix is given in 
Fig. 10.1.1. 


k 1 A ] i t: zd 
u 1 
4 22 DI СА 
a’ a aa 
Ета. 10.1.1 


We have shown by the type II supervector notation and also by a 
numerical example that the minor product moment of a matrix is sym- 
metrical. Actually, however, you have learned enough by this time to 
give a much simpler proof of this rule. You know that one definition of a 
symmetric matrix is that it is equal to its transpose. Therefore, if you 
prove that the transpose of a minor product moment is equal to the minor 
product moment, you have proved that it is a symmetric matrix. This 
you can do very simply as follows: use the rule that the transpose of a 
product is equal to the product of the transpose in reverse order. From 


this you get 
(a'a)! = а (а) (10.1.6) 


But you know that the transpose of a transpose is the original matrix, or 
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that two primes cancel each other; therefore you can write the right side of 
Eq. (10.1.6) as 


(a'a) = аа (10.1.7) 


Thus we have proved that the minor product moment of a matrix is sym- 
metric. 


The major product moment 


We can prove just as simply now that the major product moment of a 
matrix is a symmetric matrix. We do this in exactly the same fashion by 
writing 


(aa’)’ = (a^)'a' (10.1.8) 

You see that on the right side of Eq. (10.1.8) we have reversed the order of 
the factors and taken their transposes. From Eq. (10.1.8) we get 

(aa’)’ = aa’ (10.1.9) 


Thus we have proved that the major product moment of a matrix is equal 
to its transpose, or that it is a symmetric matrix. Using the matrix from 


Fig. 10.1.1 again as a numerical illustration, we can get the major product 
moment as in Fig. 10.1.2. 


matrices may be demonstrated by means of subscripts. For example, let 
the original matrix be vertical, with n rows and m columns. Then we can 


write the original matrix as in Eq. (10.1.10) and the transpose as in Eq. 
(10.1.11). 


а = anm (10.1.10) 


, 


а = Any (10.1.11) 
We would then have for the minor product moment 


а@ = аа (10.1.12) 
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But the rule for the order of a product tells us that the first subscript of 
the first factor is the number of rows in the product, the last subscript in 
the last factor is the number of columns. We see from Eq. (10.1.12) that 
the minor product moment matrix is of order m, or the width of the original 
matrix. 
From Eqs. (10.1.10) and (10.1.11), we can write the major product 
moment as 
aa! = AnmOmn (10.1.13) 


Equation (10.1.13) shows that the major product moment is of order 
n X n, or the same as the height of the original matrix. 


10.2 The Inequality of Major and Minor Product Moments 


We should make clear that the minor and major product moments of a 
matrix are, in general, not the same. It is obvious that the two cannot be 
equal if the original matrix is not square. In such a case, as you have 
already seen, the minor and major product moments are not of the same 
order; therefore they cannot possibly be equal. But even if the matrices 
аге square, two product moments are not in general equal. The general 
rule is given in Fig. 10.2.1. 

aa’ = a'a 


Fig. 10.2.1 


It is easy to prove the inequality of the two product moments even for 
Square matrices. You recall, of course, that the natural order of a square 
matrix is not automatically defined. Suppose, however, that we let n equal 
m so that we arbitrarily express the minor product moment of the matrix 
in Eq. (10.1.4). To get the major product moment, we go back to the type 
II supervector representation of a matrix and define a in terms of а column 
Vector as in Eq. (10.2.1). 
+ 
а. 


, 


в. (10.2.1) 
$T 


Then from this the transpose of a would be given as 
a’ = [а. а. ... аһ (10.2.2) 


F'rom Eqs. (10.2.1) апа (10.2.2) we get the major product as in Eq. 10.2.3. 


aa’ =| | [а. а. ... Gala (10.2.3) 
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Multiplying out the right side of Eq. (10.2.3), we get Eq. (10.2.4). 


‚ , , 
аа. а). ... ауа». 
7 Й 7 
аа = | 9 9:9. 5 аа. (10.2.4) 
xA Chur uA кез 
db. di. “йш ces Anan. 


Now going back to Eq. (10.1.4), you see that the ijth element of the minor 

roduct matrix c is given by v 

i су = aa; (10.2.5) 

You also see that the ijth element of the major product matrix in Eq. 

10.2.4) is given by н 

PE j fa = aka; (10.2.6) 
fü 1.43. 


But the right-hand sides of Eqs. (10.2.5) and (10.2.6) cannot in general ре 
the same, for їп Eq. (10.2.5) we have the minor product of column vectors 
and in Eq. (10.2.6) we have the minor product of row vectors. Therefore, 
since the elements of the minor and major product moment matrices zr 
not equal, the matrices themselves cannot in general be equal, as ers 
in the rule, Fig. 10.2.1. It is important that this distinction be kept a 
mind, for in using matrix equations to analyze large masses of data, bo iR 
of these products are frequently required, and it is very easy to confus 
them. A numerical example will emphasize the difference between them 
When the matrices are square; the difference is clearly brought out by 126: 
10.1.1 and 10.1.2 fora rectangular matrix. Even for a square matrix, gd 
we premultiply a by its transpose to get the c matrix in Fig. 10.2.2, pe 
result is quite different than when we postmultiply it by the transpose 
get the f matrix in Fig. 10.2.3. 


L1 2 i| 


3 2 6 9 9 
S Lb xg qo Les 26 20 
2 2 8]l[T.4 $ 90. 20^ 17 
a a c 
Fic. 10.2.2 
L S72 2-4 14 9 19 
22 ZS а |. = 9 9 12 
LASIS 2 s 19 12 26 
а а! f 
Fia. 10.2.3 
10.3 A Symmetric Matrix Pre- and Postmultiplied by a Matrix and Its 
Transpose 


A Р : j- 
Any symmetric matrix that is premultiplied by a matrix and postul 
plied by the transpose of the matrix is a symmetric matrix. For example 
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if we let the symmetrie matrix be s, then we can write 
(asa)! = (a')'s'a' (10.3.1) 


But remembering that two primes cancel each other and the transpose of a 
symmetric matrix is the matrix itself, we can write the right side of Eq. 
(10.3.1) as 


(asa’)’ = asa’ (10.3.2) 


In the same way, we can prove that if a symmetric matrix is postmultiplied 
by a matrix and premultiplied by its transpose, the triple product is a 
symmetric matrix. First we write 


(а'ѕа)' = a's'(a^)' (10.3.3) 
But from this we get 
(a/sa)' = a'sa (10.3.4) 


Equations (10.3.2) and (10.3.4) state that if a symmetric matrix is multi- 
plied on either side by another matrix and its transpose, the triple product 


ls à symmetric matrix. 


10.4 Square Products of Matrices 


The factors of a square product 


You have seen that the product moment of a matrix is a square matrix 
whether it is the major product or the minor product. Obviously, of course, 
a matrix and its transpose are not the only factors that yield a square 
product, but if the product of a matrix and another matrix 2 square, we 
can draw some interesting conclusions. Since the order of the product of 
two matrices is determined by the distinct dimensions of the factors, we 
know that a square product comes from two factors whose distinct dimen- 
Sions are equal. We also know that, in order for the product of two matrices 
to exist, they must have a common dimension. We know, therefore, that 
if the product of two factors is a square matrix, the height of the prefactor 
is equal to the width of the postfactor, and the width of the prefactor is 
equal to the height of the postfactor. Suppose then that z and y are two 
different matrices. If now zy’ is a square matrix, c, let us say, then and 
only then does the product z^y exist; and the product 27у is also a square 
matrix. We can put it another way by saying that if both the products 
zy’ and z^y exist, then each of these products is square, irrespective of the 
order of z. We also know that if ay’ and z'y exist, the orders of x and y 
must be the same. This principle in diagrammatic form is shown in Fig. 


10.4.1. 
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b a a 
Т | | a 
a r =a u 


Fie. 10.4.1 


Figure 10.4.1 shows clearly that if the product matrix и is of order a, then 
the height of z must be a and the width of y must be a. Now in Fig. 10.4.2, 
you see that if the height of x is a and the width of y is a we can take the 


transpose of each of these in the same order and they will have the common 
dimensions a and distinct dimension b. 


а b b 
b x a y = b v 
Fig. 10.4.2 


Major and minor products 


It is sometimes convenient to talk about the major and minor products 
of two matrices when both distinct orders are greater or less than thelr 
common order. But if the major product zy’ exists, the transpose of this 
major product, which would be yz’, also exists, Both of these are major 
products. Similarly, if the minor product т'у exists, its transpose, У i 
exists. We see, therefore, that if two matrices z and y have both à majo 
and minor product, the matrices are of the same order and have two ш 
and two major products. Conversely, if two matrices are of the pu 
order, they have two major and two minor products. For examples of we 


various types of products, we can let x and y be the matrices given in Fig 
10.4.3. 


1 2 9 2 
z-2|3 1 y=|1 2 
2 1 qe 


Fig. 10.4.3 
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Then one of the major products will be c, let us say, as in Fig. 10.4.4. 


l 2 8 5 5 
xg =g d E : 1 9 5 10 
(ee T 

, c 


2^5] 
x y = 
Fig. 10.4.4 
The transpose of this major product is shown in Fig. 10.4.5. 
. P3]nsz].|$ 3 í 
yr =|1 2 ttl 5 54 
3 5 10 7 
y Td = e 
Fig. 10.4.5 


We also have two minor products. One of these is given in Fig. 10.4.6. 


OE S К iE 
"Es те 8 9 
A y F 


Fie. 10.4.6 


The transpose of this minor product is shown in Fig. 10.4.7. 


y! К 1 | ; «In al 
t= z 
mm ţi UNE 
y z 


Fie. 10.4.7 


10.5 The Trace of a Matrix 


An important feature of the square matrix is commonly called by 
mathematicians the trace of a matrix. By the trace of a matrix, we mean no 
More than the sum of its principal diagonal elements. Since, properly 
speaking, we talk about diagonal elements only with reference to square 
matrices, we conclude that only square matrices have traces. The method 
used by mathematicians to indicate the trace of a matrix a is tr a. Thisisa 
convenient shorthand notation; however, we shall examine in detail traces 
of special kinds of products and how these can be expressed in terms of 


Matrix operations. 
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The diagonal of a matrix 


First, as you learned in Section 3.4, it is convenient to use à symbol d 
indicate the diagonal matrix made up from the corresponding principa 


diagonal elements of à square matrix. This we frequently do by using D 
with the matrix symbol as a subscript, as in Fig. 10.5.1. 


ап AR ... Gin ап 
а= а а азм D ax 
аһу Am Amm, Amm, 
Fic. 10.5.1 


A numerical example of a square matrix and its diagonal matrix is Fig. 
10.5.2. 


243 
bild. d D, = 4 
2 3-1 1 
Fie. 10.5.2 
The diagonal of a product moment 


Ап example of the diagonal matrix for the minor product moment given 
in Eq. (10.1.4) would be 


Fig. 10.5.3. 
азал 
Date аза» 
ама, 
Fig. 10.5.3 
An example of the diagonal matrix for the major product moment given 10 
Eq. (10.2.4) is Fig. 10.5.4. 
ара, 
Du = аз а». 
[A 
Fie. 10.5.4 


The diagonal of a major product 


Let us now consider the diagonal matrix for the major product of two 


matrices a and b. The first major product we shall consider is ab’. Let US 
represent a and b’ by type II supervectors as in Fig. 10.5.5. 
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a2] b um de il. Dude 


Fic. 10.5.5 


Then the major product is given in Eq. (10.5.1). 


ajb, аЬ. ... abn. 
apes ajb, а. ... abn. (10.5.1) 
TOT NAP ER 
The diagonal matrix is then given in Eq. (10.5.1). 
aj bi. 
Dim а}. (10.5.2) 
AS а, bn. 


Let us now take the transpose of Eq. (10.5.1). Here the diagonal elements 
are still the same except that the order of the factors is reversed. Equation 
(10.5.3) is the transpose of Eq. (10.5.1). 


bm. Da. ... bias. 
bdo ba, Ба. ... аһ. (10.5.3) 
m. E эст ы. 


Then the diagonal of the matrix in Eq. (10.5.3) is given in Eq. (10.5.4). 


bi ai. 
2 
Dies а. D (10.5.4) 


, 
bnan. 


But corresponding elements in the right sides of Eqs. (10.5.2) and (10.5.4) 
are the same because one is the transpose of the other. They are scalar 
products of vectors, and the transpose of a scalar is the same scalar. There- 
fore, we can write 


Dw = Dav (10.5.5) 


Equation (10.5.5) says that the diagonal matrix of a square product is equal 
to the diagonal matrix of the transpose of the square product. This, as you 
can well see, is true in general of all square matrices. The diagonal of any 
Square matrix is equal to the diagonal of its transpose. 


176 SIMPLE MATRIX COMPUTATIONS 


The diagonal of a minor product 


We may now consider the minor product of two matrices. Instead of 
using the notation in Fig. 10.5.5, let us express a as a type II row super- 
vector and b' as a type II column supervector. We then have the matrices 
expressed as in Fig. 10.5.6. 


bp а= (ai a2 us 


Fig. 10.5.6 


We can express one of the minor products of b and a by Eq. (10.5.6). 


bias bags. 2. bias 
bas bay ba. ... Ыза m (10.5.6) 
лад бааа сс Ва 


Then the diagonal matrix of the product is as given in Eq. (10.5.7). 
baa 
р.-| 8. (10.5) 


[RC 


Thus, corresponding to the case of the major product, the diagonal of the 
minor product of two matrices is equal to the diagonal of the transpose of 
the minor product of those two matrices. This is expressed as 


Dv; = D, (10.5.8) 


We may take а numerical example of Eq. (10.5.5) from Figs. (10.4.4) 
and (10.4.5). From these two figures we get Fig. 10.5.7. 


8 8 
D = 5 D, = 5 
7 7 


Fig. 10.5.7 


An example of Eq. (10.5.8) can be obt: 
From these we get Fig. 10.5.8. 


11 11 
D, e a 
| | Dy | | 


Fie. 10.5.8 


ained from Figs. (10.4.6) and (10.4.7). 


N 
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Computational notation for trace 


We have defined the trace of a square matrix as the sum of its diagonal 
elements. Therefore, if we premultiply the diagonal of a square matrix by 
a row unit vector and then postmultiply by a column unit vector, we have 
the sum of its diagonal elements. This is illustrated in Fig. 10.5.9, where 
we use the diagonal matrix D in Fig. 10.5.2. 


2 0 O}11 2 
E 11104 07 = [|1 1 14| =7 
001111 1 
Fie. 10.5.9 


It is easy to see i Fig. 10.5.9 that the answer 7 is actually the sum of the 
diagonal elements of the matrix. 
Now let us go back to Eq. (10.5.2). We can express the sum of the 


diagonal elements as 


Del = аһ. + @Ь. +... abs. (10.5.9) 


In Eq. (10.5.9) we have on the right side merely the sum of the diagonal 
elements of the matrix in Eq. (10.5.1). In the same way we can give the 
sum of the elements on the right side of Eq. (10.5.7) as 


рь = bhaa + Уза +... + mdm (10.5.10) 


Now let us look at the right side of Eq. (10.5.9) and see just what it is 
made of. The first term on the right is the minor or scalar product of the 
first row from a by the first row from b, which means that it is the sum of 
Products of corresponding elements from the first row of each matrix. The 
Second term on the right side of Eq. (10.5.9) is the scalar product of the 
Second row of a and the second row of b. This means that it is the sum of 
the products of corresponding element from the second rows of a and b. 

о for each of the remaining terms in Eq. (10.5.9), we have the scalar 
Product of corresponding rows of a and b, or the sum of products of cor- 
Tesponding elements from the two rows. Therefore, the sum of all of these 
Scalar products on the right side of Eq. (10.5.9) would be simply the sum 
of the products of all corresponding elements in the matrices a and b. Let 
US go back to Figs. 10.4.4 and 10.5.7 to see how these diagonal elements 
are actually made up in a numerical example. The first diagonal element 
of the product in Fig. 10.4.4 is 8. This is the scalar product of the first row 
of x and the first row of y (the first column of y’). In the same way, 5 comes 
from the second row of x and the second row of y, while the third element, 
7, comes from the third row of x and the third row of y. We can illustrate 
the composition of these elements by Fig. 10.5.10. 


178 SIMPLE MATRIX COMPUTATIONS 


8=(1х2)+(2х3) 
5=(3X1)+ (1x2) 
7=(2X3)+(1x1) 
20 

Fie. 10.5.10 


But now if we add together all of the diagonal elements represented in Fig. 
10.5.10, we simply have the sum of the product of all corresponding 
elements in the two matrices. For this particular case, of course, the sum 
is 20. 

Now let us see what the right side of Eq. (10.5.10) consists of. We see 
that the first term is the scalar product of the first column of b and the 
first column of a. In general, the ith term is the product of the ith column 
of b and the ith column of a; therefore, since each of these scalar products 
is the sum of the product of corresponding elements, then the sum of all of 
the scalar products would be the sum of the product of all corresponding 
elements in the two matrices a and b. We can take а numerical example 


from Figs. 10.4.6 and 10.5.8. Figure 10.5.11 illustrates the composition of 
the two diagonal elements. 


П = (1 X 2) + (3 X 1) + (2 x 3) 
9 = (2х3) +(1х 2)+(1X 1) 


20 
Fic. 10.5.11 


If we add together the two diagonal elements 11 and 9, we get 20. This is 
the sum of the products of all corresponding elements of the two matrices. 
We see, therefore, that this is the same as Fig. 10.5.10. We also see by п0% 
that since the right side of Eq. (10.5.9) is the sum of the product of all 
corresponding elements of the matrices a and b, and the right side 0 
(10.5.10) is also the sum of the products of all such corresponding elements, 
then the right sides of Eqs. (10.5.9) and (10.5.10) are equal. Therefore, 
the left sides of these equations must also be equal, and we can write 


l'Dajl = 10,1 (10.5.12) 
Equation (10.5.12) says that the trace of the major product of two matrices 


is equal to the trace of the minor product of those two matrices. 10 


summarize: the trace of a square product of matrices is the sum of products 
of their corresponding elements. 


The major and minor product moments of a matrix are special cases of 
the major and minor square products of two matrices. We can therefore 
write Eq. (10.5.13) as a special case of (10.5.12). 


10,1 = 10,1 (10.5.13) 


SPECIAL MATRIX PRODUCTS 179 


This says that the trace of the minor product moment matrix is equal to 
the trace of the major product moment matrix. A numerical example of 
Eq. (10.5.13) may be taken from the matrices in Figs. 10.1.1 and 10.1.2. 
The diagonal matrices for their minor and major product moments are 
given in Fig. 10.5.12. 


10 
14 


Fic. 10.5.12 


The trace of the minor product moment is given in Fig. 10.5.13. 


UD = [1 j” ul H - 35 


Fie. 10.5.13 
The trace of the major product moment is given in Fig. 10.5.14. 
10 1 
Dal =A 1 1 17 1|235 
8111 
Fic. 10.5.14 


aceis35. To summarize: the trace 


You see that in both these figures the tr 
he sum of squares of the matrix 


of a product moment of a matrix is t 
elements. 


10.6 Products Involving Vectors 


es and vectors are factors have special 


Produets in which both matric 
gnize at once. 


characteristics, which you should learn to reco} 


Premultiplication by a vector 

multiplies а matrix must be a row vector. 

D lied by a vector, the product is a row vector. 
ird, the order of the product vector depends only upon the width of the 

mart and not at all upon the order of the vector by which it is premulti- 

blied, These three rules follow from what we already know about order and 
е 1 i i 

“a of matrices. They are illustrated by the diagram in Fig. 


5 First, any vector that pre 
econd, if a matrix is premultip 


180 SIMPLE MATRIX COMPUTATIONS 


a b b 
Ur 


Fie. 10.6.1 


Here we have an a X b matrix z premultiplied by a row vector of order a. 
Notice that it must be of length a to be conformable with x. The product 
yields a vector in row form of length b, or of length equal to the width of 
the matrix z. A numerical example is given in Fig. 10.6.2. 


4 2 

2 i 3]|1 3|= 5 16] 
2 3 
Fie. 10.6.2 


Notice that here the prefactor is a row vector of length equal to the height 


of the matrix, and the product is a row vector equal to the width of the 
matrix. 


Postmultiplication by a vector 


The rules for postmultiplying a matrix by a row vector can also be 
deduced from the rules of multiplication you already know. First, any 
vector that postmultiplies a matrix must always be a column vector. 
Second, if a matrix is postmultiplied by a column vector, the product is ® 
column vector. Third, the order of the product vector is determined by 
the height of the matrix and has nothing to do with the order of the multi- 
plying vector. The diagram in Fig. 10.6.3 illustrates these rules. 

a 


Fig. 10.6.3 


A numerical example is given in Fig. 10.6.4. 
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4 2 14 

| BEE 

2 8 13 
Fic. 10.6.4 


Here we have a 3 X 2 matrix postmultiplied by а second-order column 
vector, which gives a third-order column vector. 


Pre- and postmultiplication by a vector 


If we both pre- and postmultiply a matrix by a vector, we always get a 
scalar quantity. Because of the rules of order or dimensionality for multi- 
plication, the first vector must always be a row vector, and the last vector, 
or postfactor, must always be а column vector. We can illustrate the 
general principle in Fig. 10.6.5, where V, is one vector and V» another 
vector, and X is а matrix. 

VXV, = a scalar quantity 
Fie. 10.6.5 
he left of X in Fig. 10.6.5 is primed to 


You will notice that the vector on t 
V, on the right is unprimed to show 


show that it is a row, while the vector 
that it is a column. 

A numerical example of pre- 
vectors can be constructed from Figs. 
same matrix and also the same vector 
We have then Fig. 10.6.6. 


4, 2 
2 

р 1 8113 
|: 18 

Fig. 10.6.6 


We could first get the product of ће first two factors and then multiply 
by the last factor. But the product of the first two factors 18 already 
duct as in Fig. 10.6.7. 


given in Fig. 10.6.2, so we may write the pro 


(15 16] E = 30 + 48 = 78 
Fra. 10.6.7 


We could also get the product of the last two factors in Fig. 10.6.6 as in 
Fig. 10.6.4, and then premultiply this column vector by the first factor, or 
row vector. This we do in Fig. 10.6.8. 


14 
(2 1 af = 28 + 11 + 39 = 78 
13 


Fic. 10.6.8 


and postmultiplication of а matrix by 
10.6.2 and 10.6.4. Let us use the 
s for pre- and postmultiplication. 
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In either case, the result is the same, as we knew it would have to be, 
because of the associative law of multiplication. 


10.7 Multiplication of a Matrix by a Unit Vector 


The unit vector plays a much more important role in the analysis of 
behavioral science data than in the more purely mathematical treatment 
of matrix algebra. We shall, therefore, consider the most important cases 
of multiplication of matrices by unit vectors. 


Premultiplication by a unit vector 


The premultiplication of a matrix by a unit vector yields a row vector 


whose elements are the sums of the elements of corresponding columns. À 
numerical example is given in 


2 4 
fl 1 1/8 2|=[6 9 
1 3 


Fic. 10.7.1 


You see that on the right side of Fig. 10.7.1, the first element in the row 
vector is 6, and it is the sum of the elements in the first column of the 
matrix. The second element is 9, and it is the sum of the elements in the 
second column of the matrix. Therefore, if we want to sum the columns of 
a matrix we premultiply it by a row unit vector. 


Postmultiplication by a unit vector 


The postmultiplication of a matrix by a unit vector gives a column 
vector whose elements are the sums of the elements in the corresponding 
rows of the matrix. A numerical example is given in Fig. 10.7.2. 


2 4 1 6 

JEE 

1-8 4 
Fie. 10.7.2 


You see that the first element in the column vector on the right side in 
Fig. 10.7.2 is 6. This is the sum of the elements in the first row of the 
matrix. The second element, 5, is the sum of the elements in the second 
row of the matrix, namely 3 and 2; and the third element, 4, is the sum of 
the elements in the last row of the matrix. "Therefore, to sum the elements 
in the rows of а matrix, we postmultiply the matrix by a column unit 
vector. 
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Pre- and postmultiplication by a unit vector 

If we pre- and postmultiply а matrix by unit vectors we get а scalar 
quantity that is the sum of all of the elements in the matrix. Using the 
Same matrix as in Figs. 10.7.1 and 10.7.2, we pre- and postmultiply it by 
unit vectors as in Fig. 10.7.3. 


2.4 1 

[А 113 2 Н 
11:8 
Fic. 10.7.3 


Here we can first multiply together the first two factors and then multiply 
by the last, or we can first multiply the last two factors and then premulti- 
ply by the first. We shall use the former method. Already in Fig. 10.7.1 
the first two factors have been multiplied together. We use this product 
and postmultiply it by the unit vector as in Fig. 10.7.4. 


(6 9] И zl 


Fic. 10.7.4 


You can verify, as shown in Fig. 10.7.4, that the sum of the elements in the 


matrix of Figs. 10.7.1 and 10.7.2 is 15. 


10.8 Multiplication of a Matrix by an e; Vector 


science data, we fre- 


In the application of matrix algebra to behavioral 
vector. The rules for 


quently have occasion to multiply а matrix by an ei 
Such multiplication are very simple and useful. 


Premultiplication by an e; vector 


A matrix premultiplied by an e; vector куе 
TOW of the matrix. Thus 
ex = x (10.8.1) 


a vector which is the ith 


A numerical example for Eq. (10.8.1) is Fig. 10.8.1. 
135 
2. 6.4 
gauge? 
4 8 2 


Fie. 10.8.1 


. You see that in Fig. 10.8.1, ¢ is e, and therefore the product on the right 
is the second row vector of the matrix. 
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Postmultiplication by an e; vector 


Postmultiplication of à matrix by an e; vector gives the ith column 
vector of the matrix. Thus 


Хе; = X4 (10.8.2) 
A numerical example of Eq. (10.8.2) is Fig. 10.8.2. 
13 5 5 
2 6 4 4 od 4 
Sc wd 1 6 
43 2 2 
Fic. 10.8.2 


Here e; is an es column vector; therefore, the product is the third column 
vector of the matrix. 


Pre- and postmultiplication by an e; vector 


Pre- and postmultiplication of a matrix by an е! vector and an e; vector 


respectively gives a scalar quantity which is the ijth element of the matrix. 
Thus 


ехе = X (10.8.3) 
A numerical example of Eq. (10.8.3) is Fig. 10.8.3. 
S 
p 198.54 “ies 
A3 9 
Fic. 10.8.3 


10.9 Products Involving Diagonal Matrices 


Diagonal matrices play an important role in many applications of 
matrix algebra. It is, therefore, well to know the most important rules 
governing their products with other matrices and with one another. 


Premultiplication by a diagonal matrix 


If we premultiply any matrix by a diagonal matrix, the product 18 ® 
matrix such that each element in a given row is the product of the a: 
responding element in the original matrix and the diagonal elemen 
corresponding to that row from the diagonal matrix. A numerical example 
of premultiplieation by a diagonal matrix will clarify this rule. In Fig. 
10.9.1, we show a matrix premultiplied by a diagonal matrix. 
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3°20) 0 4-9 41 
iS OG 32-35 1 = с 
O OA 8 4 1 
р а 
Fic. 10.9.1 


ements of c as the scalar products of the 
corresponding row vectors from the prefactor in Fig. 10.9.1 and the ap- 
propriate column vector from the postfactor. Taking each of the elements 
of c in turn, we get the results in Fig. 10.9.2 for the first row of c. 


on = (8X2) + (0х6) + (0х 2) = 3 х2 
се = (3х4) + (0х2) + (0x3) = 3х1 
cn = (3х3) + (0X5) + (0х4) 3X3 
= (3х1) + (0х2) + (0х1) = 3х1 
Fic. 10.9.2 
e can show in the same way that 
he elements in the third row. 


Now let us express each of the el 


Ci 


For the second row of the elements of c, w 
they are given by Fig. 10.9.3, as are also t 
ty = 2X6 c&n = 2X2 in = 2X5 Cu = 2X2 
cy = 4X 2 cy = 4X3 сз = 4X4 си = 4X1 

Fie. 10.9.3 


Now we put together the elements of Figs. 10.9.2 and 10.9.3 to constitute 


the entire c matrix in Fig. 10.9.4. 
3x2 3x4 3X3 3X1 
с=| 2х6 2X2 2х5 2х2 
4x2 4x3 4x4 4X1 
Fig. 10.9.4 


But we see here that in the first row the first factor of each element is 
rix in Fig. 10.9.1. The 


always 3, or the first element in the diagonal mat 
Second factor of each element in the first row is always the corresponding 
element from the right-hand matrix of Fig. 10.9.1. The same is true of 
rows 2 and 3 except that the constant multipliers now are the corresponding 
elements 2 and 4 from the diagonal matrix. 

The general rule for the premultiplication of а matrix by & diagonal 


matrix is given in Eq. (10.9.1). 


D, merum Qn da +--+ Mm 
0* D esr ба. йа £e OB | 
0 ора | ast ас zc Rem 
Dian Dian ... уал 


D Раз ... Daten 
2021 i 202, (10.9.1) 


Daam Dnran ... DnOnm 
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The rule for premultiplication by a diagonal matrix is further emphasized 
by the diagram in Fig. 10.9.4. 


EM — 


Fic. 10.9.4 
Postmultiplication by a diagonal matrix 


If à matrix is postmultiplied by a diagonal matrix, the product is the 
original matrix with each element in a given column multiplied by the 
corresponding element in the diagonal matrix. Figure 10.9.5 illustrates 


this rule. 
23 6|[2 оо 4 12 18 
1 5 2||0 4 0[- [2 20 6 
42:810903 8 8 9 
a D € 
Fie. 10.9.5 


Here you see that the first column of c on the right is obtained by multiply- 
ing each element in the first column of a on the left; by the first element 1n 
D, which is 2. So for the second column in e, each element in the secon 
column of a has been multiplied by 4, the second element in the diagonal 
matrix. The third column is similarly obtained. 


Equation (10.9.2) illustrates the general rule for postmultiplying ? 
matrix by a diagonal matrix. 


а а ... а |р 0 ... д 

Qu бы o am 0 Dr i. 0 x 

Cit би c» Gan 0 ПЕЕ Dp, 
ару ар ... аы 
Dı ањ, ... аһ (10.9.2) 
аР ар, ... dnmDm 


This гше for postmultiplication by a dia, 


gonal matrix is emphasized further 
by Fig. 10.9.6. 
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МИШ 


Ета. 10.9.6 


Pre- and postmultiplication by a diagonal matrix 

If a matrix is pre- and postmultiplied by a diagonal matrix, the product is 
the matrix whose ijth element is the triple product of the ith element 
from the left, diagonal multiplier, the ijth element of the matrix, and the 
jth element of the right diagonal multiplier. Figure 10.9.7 illustrates this 
rule, 


2 0 o]2 з 2x2x1 2X83X5 4 30 
030 "n ?|-|sxixi sxsxs|- 3 75 
0 o alla 2]19 54 [ахах 4x2x5] Li6 40 


Fie. 10.9.7 
Equation (10.9.3) illustrates the general rule for pre- and postmultiplying 
a matrix by diagonal matrices. 
Di Ou 0 аһ Qo Um di 0 0 
DG. |l «o mee || 9. dk os P Ius 
Of alla. me уша ШП 9 d, 
Didan Didar ... Didmaım 
Dedian Гьфаљ + + + Dodmaom (10.9.3) 
Юша Dd ... Dndmanm, 


(10.9.3) we have placed 


Note that in the elements on the right side of Eq. 
because scalar multi- 


the d,’s before the a;;’s. This we can do, obviously, 
Plication is commutative. 


Multiplication of a unit vector and a diagonal matrix 


À vector may always be expressed as а produet of a diagonal matrix and 
a unit vector. For example, any row vector can always be expressed as а 
Tow unit vector postmultiplied by а diagonal matrix. This is illustrated in 


Eq. (10.9.4). 
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E 
o 
oo 


[o ЖЕКА, em Te ees B COR ИЕ (10.9.4) 
0 Ü uni Be 
A more concise notation for Eq. (10.9.4) would be 
Vi cg, (10.9.5) 


This means that а row vector may be expressed as the product of a unit 
row vector postmultiplied by a diagonal matrix whose diagonal elements 
are the same as the elements in the row vector. 


A numerical example of Eqs. (10.9.4) and (10.9.5) is given in Fig. 10.9.8. 
2 0 0 
2 з 7=f1 1 ulo 3 0 
Bu 7 
Fia. 10.9.8 
In the same way, any column vector may be expressed as the product of 


a unit column vector premultiplied by a diagonal matrix whose elements 


are the same as the corresponding elements in the vector, This rule may 
be illustrated by Eq. (10.9.6). 


bi РО О 


1 
b, ОЕ C 1 (10.9.6) 
b, ОБО be i 
It may be expressed more compactly as 
Y, = Dsl (10.9.7) 


A numerical illustration of this rule is Fig. 10.9.9. 


4 4 0 Ol[1 

2|=|0 2 Offi 

1 0 0 1]|1 
Fic. 10.9.9 


10.10 Products in Which All the Factors Are Diagonal Matrices 
The product of two diagonal matrices 
The product of two diagonal matrices is a diagonal matrix each of whose 


diagonal elements is the product of the corresponding elements of the 
factors. This rule is illustrated in Fig. 10.10.1. 
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3 00][2.0 0 3х2 0 0 600 

010[[0 3 о |= 0 1x3 0 =|0 3 0 

0 0 2][0 0 4 0 0 2X4 00 8 
Fie. 10.10.1 


It is evident that this rule must hold. We have seen that to premultiply 
a matrix by a diagonal matrix amounts to multiplying each element in the 
given row of the matrix by the corresponding diagonal element in the 
diagonal matrix. But since in this particular matrix we have a diagonal 
matrix premultiplied by а diagonal matrix, then all elements in а given 
row would be 0 except the diagonal ones, and these, in turn, would be 
multiplied by the corresponding element of the other diagonal matrix. 
Using the rule of postmultiplication by a diagonal matrix in which the 
columns of the matrix are multiplied by the corresponding elements of 
the diagonal matrix, we again get the same results, namely, that all 0 
elements in the prefactor still remain 0. The rule for the product of two 
diagonal matrices is given by Eq. (10.10.1). 


а 0 €—— 0 bi 0 eet 0 abı 0 TE 0 

0. у ISO tt a 40 O mess 0 

CAE Oa ices са И ONE р 
(10.10.1) 


This can be indicated in compact notation as 
DaD = D. 


We understand, of course, that in Eq. (10.10.2) all three matrices are 
diagonal. Then any particular diagonal element of the matrix De, вау €i 
сап be expressed as the product of the corresponding diagonal elements 


from the matrix D, and Dy. Thus 


(10.10.2) 


cd (10.10.3) 


The product of any number of diagonal matrices 


Knowing the rule for the multiplication of two diagonal matrices, we can 
easily state the rulefor the product of any number of diagonal matrices. This 
Tule is that the product of any number of diagonal matrices 1s a diagonal 
Matrix each of whose diagonal elements is the product of the corresponding 
diagonal elements of the factors. This rule must follow because no matter 

Ow we apply the associative law in forming the product of more than two 
diagonal matrices, we shall always have at each step the product of two 
diagonal matrices to compute. An example of the product of three diagonal 


Matrices is given in 
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зо ooe 0 O0|[1 0 O 120 0 
0 1: 0]0 38 оо 2 01| = O & O0 
0.0 2][0 0- 2]|[0-.0. 4. 0 0 16 


Fra. 10.10.2 


Now let us indicate the product of any number of diagonal matrices by Eq. 
(10.10.4), where again each of the matrices in the equation is diagonal. 


D.D»... D: = Dx (10.10.4) 


Then the ith diagonal element of the product matrix, or the element ci, may 


be expressed as the product of the corresponding elements of the matrices 
on the left of Eq. (10.10.4). 


Ci = aby... 2; (10.10.5) 
The commutative law and diagonal matrices 


Although matrix multiplication is in general noncommutative, clearly 
all diagonal matrices of the same order are commutative one with another. 
This must be true because any element in the product matrix is the scalar 
product of the corresponding elements of the factors. The only effect of 
changing the order of the factors is to change the order of the elements in 
the scalar products that form the elements of the product matrix. But 
since all scalar quantities are commutative with respect to multiplication, 
the change in the order of the factors has no effect on the product matrix. 


For the product of three diagonal matrices, we could therefore write 
Fig. 10.10.3. 


Р.Р”, = DVD. = р.р, = D4D.Dy = DD.D, = D-DaDs 
Fig. 10.10.3 
10.11 The Powers of a Diagonal Matrix 
The general case 


Just as scalar quantities can be raised to various powers, so also can 
diagonal matrices. To raise any diagonal matrix to a given power we 


simply raise each of its diagonal elements to that power. This principle is 
stated in Eq. (10.11.1). 


GAOL e csv SOE üt ð 0 
0 a OF WO а 0 (10.11-1) 
8 0 а, ou E 


Let us now consider several special cases of this rule. Suppose we wish 
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to raise a matrix to the second zd 
И > Se Ower This would be expresse by Eq. 
(10. 11 2). р ou d d 


abe fro 

2 0 CEP TO) 
mw ME (10.11.2) 

0 0 аё 


А numerical example of this equation is Fig. 10.11.4. 


4 0 OF 16 0 0 

0:380: | m] 09 0 

Ü 0 1 0/07. 1. 
Fre. 10.11.4 


The square root 

in scalar algebra we can raise quantities 
to fractional powers. For example, we can raise the number 4 to the 1/2 
Power, which means that we take the square root of 4, which is 2. In the 
Same way, we can also take the square root and other roots of a diagonal 
matrix. Equation (10.11.3) shows how we can in 
diagonal matrix. 


But we must also remember that 


dicate the square root of a 


um $5 0 


0 үш... 0 (10.11.3) 
AG) OP «уе NOR 


are root of a diagonal matrix is given in 


р? = 


A numerical example of the squ 


Fig. 10.11.5. 
о o op [3090 
о 25 0| =|0 5 0 
0 0 36 0^0. 8 


Fic. 10.11.5 


agonal elements are negative, their 


It should be noted that if any of the di 
ook we do not make use of such 


Square roots are imaginary. In this b 
quantities. 


The inverse 
We also recall from scalar algebra that a number may be raised to a 
d to a negative power is equal to 1 divided 


negative power. A number raise 1; 
by that number raised to the same positive power. For example, 4 raised 
to the —1 power is 1/4. We can also raise а diagonal matrix to the —1 
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power simply by letting k be —1 in Eq. (10.11.1). Equation (10.11.4) 
shows how we indicate the —1 power, or, as it is called, the inverse of the 
diagonal matrix. 


1 
a) 0 0 x 0 0 
1 
pasje Ж oe бл Fae 9 ууу) 
б. б уа 0 0 x 
an 


Obviously, the a; in Eq. (10.11.4) are all assumed to be different from 0. 
Figure 10.11.6 gives a numerical example of raising a matrix to the —1 


power. 
"7 0 8I" e 39 0 
0-34 0| =) 0 .25 0 
0:30. Di 3E 


Fie. 10.11.6 


But we can also have a negative fraction for the exponent of a diagonal 
matrix. 'Тһе most common exponent of this kind which we have for 
diagonal matrices in the analysis of data matrices is —1/2. Equation 
(10.11.5) shows how we raise а diagonal matrix to the —1/2 power. 


pu eee — а 0 
а 0 Van О 
1 
cda — 0 zu ju 0 — 0 
рт! = 23 = As 
0 0 save ДЕМА 0 0 Mu 
* Van 
(10.11.5) 


A numerical example of a matrix raised to the —1/2 power is Fig. 10.11.7- 


4 0 Qu s 0 6 
0 16 0 -|0 25 0 
0 09 0 0 .33 


Fie. 10.11.7 
10.12 Matrix Products Involving Scalar Matrices 


We can multiply any matrix by a scalar matrix simply by multiply me 
each of its elements by the scalar quantity in the diagonal element of th 
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scalar matrix. Figure 10.12.1 shows how we premultiply à matrix by a 
scalar matrix. 


200 0][4 2 1 8 4 2 

0:539: D: 30: | odo |e 6 2 12 

002 02 4 7 |: € 4 

90 0 0 2]||[t 4 2586232 
Fic. 10.12.1 


This rule follows directly from the multiplication of а matrix by a diagonal 
matrix, It is simply a special case of such multiplication in which the 
elements in the diagonal matrix are all equal. Let us now postmultiply the 
matrix in Fig. 10.12.1 by а scalar matrix of diagonal elements 2, as in 


Fig. 10.12.2. 
2 
ET I Ud X 1^ 
2 04:7 9 2 B|7[|4 S 14 
0021 |26 2 
TE d 


Fio. 10.12.2 


Now you will notice that the right-hand sides of the equations in Figs. 
10.121 and 10.12.2 are identical. The only difference between the left- 
hand sides of the two equations i$ that in 10.12.1 we have premultiplied by 
a scalar matrix of elements 2, and in 10.12.2 we have postmultiplied by a 
Scalar matrix of elements 2. Notice, however, that the two scalar matrices 
аге not equal. They cannot be equal since they are not of the same order. 


10.13 Products Involving Scalar Quantities 

ewhat simpler concept than that of scalar 
lly the same purpose. Instead of talking 
Itiplication of other matrices by them, 
we can talk about multiplying matrices by scalar quantities. When we say 
that we multiply a matrix by а scalar quantity, we mean that we multiply 
each of the elements by that quantity. An example of a matrix multiplied 

10.13.1 and 10.13.2. 


We shall now consider a som 
matrices, which will serve essentia 
about scalar matrices and the mu 


by the scalar quantity 2 is given in Figs. 
&&7| [864 
A ЖАР Ies A 2 
Bio 4 4| |e 8 
43 2 8 6 4 


Fic. 10.13.1 
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4 3 7 8 6 14 
ева. 84 2 
Stop up banana ы A 
432 8 6 4 
Fig. 10.13.2 


Now in Fig. 10.13.1 we have premultiplied the matrix by the scalar quanti- 
ty 2. In Fig. 10.13.2 we have postmultiplied the matrix by the scalar 
quantity 2. You see now that we get the same results whether we рге or 
postmultiply the matrix by 2. This result leads us to the rule that in matrix 
multiplication a scalar quantity is commutative with a matrix. As you 
recall, this statement is not in general true for scalar matrices, because 
unless the matrix is Square, it cannot be premultiplied by the same scalar 
matrix used for postmultiplication, since the two are not of the same order. 


Commutativity and scalar factors 


Given the very simple rule for multiplying a matrix by a scalar quantity, 
it should be easy to see that all scalar quantities in the factors of a emer 
of matrices and scalar quantities are commutative with one another an 
with all of the matrix factors. For example, let us suppose two scalar 
quantities a and b and two matrices z and y, and consider all possible 
products of these four factors in which z precedes y. All such products 
would be equal. Some of the products are given in Fig. 10.13.3. 


abry = балу = axby = bray = brya = xbya = xayb = xyab 


Fig. 10.13.3 


product of the scalar quantities Separatel. 

You should be aware o 
multiplication by a scal 
product of matrices and 


y. trix 
f à paradox arising out of our definition of matr 
ar quantity. Suppose, for example, we have ? 
vectors as shown in Fig. 10.13.4. 


H (8 ар alle) 


Here we have the product of f 
one of which is a matrix. Su 
symbols as in Fig. 10.13.5. 


d 
our factors, three of which are vectors qud 
Рроѕе now we indicate each of the vectors 
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V, = [i] V} = [2 3] 


Fig. 10.13.5 


We could then write the product in Fig. 10.13.4 as 
Vi(ViMV;) 
Fic. 10.13.6 


Notice now that the product in parentheses here constitutes a matrix 
Premultiplied by a row vector and postmultiplied by a column vector. 
You will recall that such a product is always a scalar quantity. But we 
have said that scalar quantities are commutative with matrices. Since a 
Vector is a special case of a matrix, a scalar would also be commutative 


with vectors. Therefore, we can write Fig. 10.13.6 as 
(ИМУ з) Vi = Vi(V3M Vs) 
Fia. 10.13.7 


Or removing the parentheses, we would have 
VIMVsVi = ViV2MVs 
Fie. 10.13.8 


But on the left side of Fig. 10.13.8 the last two factors are both column 
Vectors. This is a paradox because we cannot premultiply a column vector 
by a column vector. We can, of course, multiply the first three factors 
together and get a scalar quantity. Then we can premultiply the column 
Vector V, by the scalar quantity. But clearly the associative law of multi- 
Plication does not hold on the left side of Fig. 10.13.8, because we cannot 
multiply together the last two adjacent factors. This exception to the 
associative law of multiplication comes as a consequence of the definition of 
multiplication of a matrix by a scalar quantity. It is for this reason that 
You should learn to recognize a scalar product of factors immediately in 
any matrix equation. Whenever you see a product in which a row vector 
is followed by a column vector, whether or not one or more matrices 
Separate the two, you should know at once that you have a scalar quantity 
that can be commuted with other factors in the series, and that such cases 
are an exception to the associative law of multiplication. Another simple 
example of the exception to the associative law of matrix multiplication is 


Blven in Fig. 10.13.9. 
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2 2] 2 | 
HGE [s] Е ш Hl E 
2 
ог [|вха+1 хз =вха+1›‹з{?] 
2 2 
: Je- 
kek 
jn 9} [9 


Fic. 10.13.9 


AE re 
It is particularly important that you be aware of the nonassociative uere 
of matrix multiplication involving scalar products of vectors, because 


е а ; rix 
exception is not usually pointed out by writers on the subject of mat 
algebra. 


The scalar matrix as the product of a scalar and the identity matrix 


Although the scalar matrix as such is used extensively in the theory of 
matrices, we find little use for it in the application of matrix algebra to ii 
solution of problems involving primary matrices, In multiplication, as a 
have seen, we usually substitute a scalar quantity for a scalar matrix. {б 
addition, of course, we cannot use this deviee. Ifi 
add a scalar matrix to other Square matrices of t lar 
indicate the scalar matrix as the identity matrix multiplied by а s 
quantity. Suppose, for example, that the elements of a third-order scà 
matrix are 3. We can indicate the scalar matrix as in Fig. 10.13.10. 


; T, 
t is necessary, howeve K y 
he same order, we usu: 


зоо TRUNG 

03 0|-3|0 1 0 

00 3 0 0 1 
Fig. 10.13.10 


Or in general if the elements of the scal 


; А ified 
ar matrix are simply some spec! 
scalar quantity, 


вау a, we can indicate this matrix as in Fig. 10.13.11. 


а 0 0 
0 

5 0 = al 
0 0 а 


Fie. 10.13.11 
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10.14 Multiplication by the Identity Matrix 


Product of the identity matrix with a matrix 


We shall now see why the identity matrix serves the same purpose in 
matrix algebra as the number 1 does in arithmetic and scalar algebra. In 
scalar algebra, if we multiply a number by 1, we get the same number. For 
example, 1 X 3 = 3. In the same way, if we multiply а matrix by the 
identity matrix, we get the same matrix for the product. This must be 
true because the identity matrix is a special case of the diagonal matrix, so 
that if we premultiply a matrix by the identity matrix we multiply each of 
Its rows by 1, and if we postmultiply by the identity matrix, we multiply 
each of the columns by 1. Obviously, this leaves the matrix unchanged. 
These rules we illustrate by Eq. (10.14.1), where A is the matrix and I is 
the identity matrix. 

IA=Al=A (10.14.1) 
It is important to remember, however, that the identity matrix on the left 
of Eq. (10.14.1) and the identity matrix in the middle term are not neces- 
sarily equal. The reason for this is, of course, that if the matrix A is not 
Square, these two identity matrices will not be of the same order. Figure 
10.14.1 is a numerical example of the rule we have just stated. 


100 01[2 4 2 2 4 2]n 9 o 242 
fae eee £32 8 5-1 2.5 
00.3 olla i 2[-|k f Allo: 0 4 l4 2 
ооо 11121 4 214 2. 4 


Fie. 10.14.1 


Here you see that the value of the matrix is unchanged when pre- or 
Postmultiplied by the identity matrix. You also see that in this particular 
case the identity matrix as a prefactor is not equal to the identity matrix 
as a postfactor. The prefactor is à fourth-order identity matrix, whereas 


the postfactor is a third-order identity matrix. 


Powers of the identity matrix 

Another way in which the identity matrix b 
that it is unchanged by being raised to any power, 
trated in Eq. (10.14.2). 


ehaves as the number 1 is 
say k. This is illus- 


lo et a oe Te 2 Я E qM 
г ü atus рО Dee A 
Ox dea E ME. ONE ы i 


(10.14.2) 
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'The rule can also be stated more compactly by 
= (10.14.3) 


Equations (10.14.2) and (10.14.3) are true no matter what the value of k. 
Some of the most common examples are 


Mal fay o ns (10.14.4) 


We can therefore take the inverse of I; we can raise it to the second power; 
we can take the square root or raise it to the 1/2 power, and in no case do 
we change its value. In expanded notation, Fig. 10.14.2 shows how by 
raising J to the —1 power, we do not change the matrix. 


LOB xc PE UM фу с 
RM Е rcc: 
On 05 ОТ ULT TE 
i 0 is "20 0 
1 
1 0 
os 0 029] 
1 = 
0 0 КЕ 
1 
0 0 Т 0 0 1 
Fig. 10.14.2 


10.15 Multiplication by the Sign Matrix 


Premultiplication of matrix by a sign matrix 


Premultiplication of а matr 
elements in rows correspondin, 
is given in Fig. 10.15.1. 


y " : A ll 
ix by a sign matrix changes the signs of m 
Е to the —1 in the sign matrix, An examp 


1 0 0)[4 2 4 2 

0 —1 0 |: Lo 27 

0 Oar a 3 4 
Fig. 10.15.1 


Here the second diagonal element in the sign matrix was — 1; therefore the 


: : s 
sign of the elements in the second row of the matrix was changed from plu 
to minus. 


Postmultiplication of а matrix by a sign matrix 
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4 3 1//1 0 0 d c. I 
39 5 2]||]0 =1 0|=|3 -5 – 
316110 Ü = 3 —1 -6 


Fie. 10.15.2 
Since the last two diagonal elements of the sign matrix are — 1, the sign of 
the elements in the last two columns of the matrix was changed from plus 
to minus, 


Pre- and postmultiplication of a square matrix by the same sign matrix 


; If a square matrix is pre- and postmultiplied by the same sign matrix, the 
Signs of the diagonal elements are left unchanged. To illustrate this we 
let i be a particular sign matrix, M a square matrix of the same order, and 


H the product of M pre- and postmultiplied by t. Then 
iMi = H 

The rule we have stated then says that the diagonal matrix of M is the same 

as the diagonal matrix of H. Thus 


(10.15.1) 


Dae Dy (10.15.2) 
Figure 10.15.3 is a numerical example and shows why this rule must hold. 
їр pa a aada oo 9 
От 0 ess zoh|[0o--2 9: 9). 
ooa ola vae 01-0 
O зулу [ae ооа ро 20:0 =F, 
4 38 3 МЕГ OO 39 d 2 pd 
a oe су || о тйс орев ую e п 
блуза 62) о OL. ко озш кш 
S QU UR MP FEET TS ЫШ wd 235 
Fie. 10.15.3 


u change the signs of elements 


When ulti he sign matrix у 
à you premultiply by the si ir i 
y ЭУ you change the signs of the 


, 


™ certain rows. When you postmultiply Bs 
elements in the corresponding columns, but the element that is in а row 


d va also in the corresponding colu 
€ Sign of each diagonal element is c 


mn is à 
hanged by premultiplication and then 


changed again by postmultiplication. Sin ninu: 
Plus, this means that the net result is no change in sign. 


Powers of a sign matrix 


diagonal element. Therefore, 


ce a minus times a minus is a 


, While it is possible mathematically to talk about powers of a sign matrix 
ìn which the exponent is a fraction, in this book we shall consider only 
Powers in which the exponent is a whole number. We notice first, then 

д 
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that if the exponent is a whole number k, the sign matrix to the kth power 
is equal to the sign matrix to the —kth power. If we let т be the sign 
matrix, this relationship is expressed as 


hart (10.15.3) 
An example of Eq. (10.15.3) where Ё is equal to 2 is given in Fig. 10.15.4. 
1 0 0 ор 1 0 0 op: 
0-10 o| 10 -1 0 0 
0 0: 1 ге or OF 31 0 
0 00 -1 0 00 —1 
or 
» 5 & 4 í onm @ 
1? 
1 
0 (-1) 0 0 й 0 Cay 0 0 
в m 1 
0 0 i? 0 0 0 T 0 
1 
-— 2 
0 0 0 (—1) 0 0 0 (D 
or 
L 0 0 0 1 0 0 0 
0: 3. 70; 10 0 i 0 0 
0010| |оо о 
0017 000 1 
or 
L0 6 0 LU. 00.0 
0- I 0 0| | 1.0.0 
OF 50): LU Oe Or S0 
000 1 0.0; 9T 
Fic. 10.15.4 


You will notice that in Fig. 10.15.4, we took the square on the left side of 
the sign matrix. Taking the square of a sign matrix means raising each of 
the diagonal elements to the second power. But the square of —1 is +1; 
furthermore, any even power of —1 is --1. Therefore, any sign matrix 
raised to an even power is the identity matrix. In particular, the sign 
matrix times itself is the identity matrix. This relationship is expressed as 


В = І (10.15.4) 


SPECIAL MATRIX PRODUCTS 201 


or in general, for k an even number, Eq. (10.15.5) summarizes both Eq. 
(10.15.2) and the general rule for even powers of i. 


t = 17% = [ (10.15.5) 
A numerical example of Eq. (10.15.4) is Fig. 10.15.5. 


1 0 о 0 0 i Q 
0 —1 опо —1 ожо 10 
0 0 —1lL0 QO: =i 07-0) 1 


Fig. 10.15.5 


If the exponent k of the sign matrix is an odd number, then the result is 
the sign matrix itself. This relationship can be expressed as 


ha tk at (10.15.6) 
It is easy to see why this rule must be true if we take a numerical example 
in which the exponent is 3, as in Fig. 10.5.6. 
1 0 0p 1 0 0||1 0 0||1 0 0 


0 —1 Oo —1 оно —1 00 —1 0 
0 0 —1 0 0 =1110 0 —1||0 O =], 
or 
1 0 0p E 0 Ор 0 0 1 0 0 
Q =l б MESES ro 0.70101 0|=|0 -1 0 
0 0 —1 0-30. ALO Üü cL 0 0 el 
Fig. 10.15.6 


You see that a matrix raised to an odd-number power can always be ex- 
pressed as that sign matrix raised to the next smallest even number power 
and then multiplied by the sign matrix itself. Therefore, any sign matrix 
raised to an odd-numbered power is the sign matrix itself. 

The sign matrix is used extensively in the analysis of data matrices where 
the attributes are personality traits that may be expressed positively or 
negatively such as ascendance-submission, generosity-miserliness, intro- 
version-extroversion, and so on. If certain attributes in a personality data 
matrix are to be reversed, this is done by postmultiplication with an 


appropriate sign matrix. 
10.16 Multiplication by a Permutation Matrix 
You will recall that a permutation matrix is obtained from an identity 


matrix by interchanging certain rows or certain columns. Therefore, in a 
permutation matrix each row and each column has only a single 1 in it and 
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all the other elements are 0. As we shall see, multiplication by a permuta- 
tion matrix has two distinct results, depending on whether we pre- or 
postmultiply the matrix by the permutation matrix. 

Any permutation matrix may always be regarded in one of two ways, 
that is, as having been obtained from the identity matrix by interchanging 
either certain columns or certain rows of the identity matrix. 


Postmultiplication by a permutation matrix 


If we postmultiply a matrix by a permutation matrix, we look upon the 
permutation matrix as having been obtained by interchanging certain 
columns of the identity matrix. If then we postmultiply a matrix by а 
permutation matrix, we get a matrix that is the same as the original except 
that the columns have been interchanged in exactly the same way as the 
columns of the identity matrix were interchanged to get the permutation 
matrix. This is illustrated by Fig. 10.16.1. 


2 41 5 oW 
ра БКО 
14 61/1, 9 of [9 1 4 
6 5 2 2-8 5 
a T b 
Fic. 10.16.1 


Here you see that to get the permutation matrix, the last column of an 
identity matrix was put as the first column. Therefore, in the product 5, 
the last column of the matrix a becomes the first. The first column of the 
identity matrix was put second; therefore, the first column of the matrix a 
becomes the second in the matrix b. The second column of the identity 
matrix was put last in the permutation matrix; therefore, the second 
column of the matrix a becomes the last in the product matrix b. 


Premultiplication by a permutation matrix 


If we premultiply a matrix by a permutation matrix, we look upon the 
permutation matrix as having been obtained by interchanging certain rows 
of an identity matrix. Then the product of a matrix premultiplied by as 
permutation matrix gives the same matrix with rows interchanged in 
exactly the same manner as the rows of an identity matrix were inter- 
changed to get the permutation matrix. An example of premultiplication 
by а permutation matrix is given in Fig. 10.6.2. 


0 0 1 Olf? 4 i 145 
0 1 OG o[||3 2 3 8 $3 
OO 0 Tit xm ollie 5 9 
тоо 0]|65 2 2 4 1 


Fic. 10.16.2 
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Here you see that the third row of an identity matrix was put first in the 
permutation matrix; therefore the third row of the matrix a becomes the 
first in the product matrix b. The second row of the identity matrix 
remains unchanged in the permutation matrix; therefore the second row 
of the matrix a remains unchanged in the second row of the matrix b. The 
fourth row of the identity matrix was put third in the permutation matrix; 
therefore, the fourth row of the matrix a was put third in the product 
matrix. The first row of the identity matrix was put last in the per- 
mutation matrix; therefore the first row of the matrix a was put last in the 


product matrix. 


Pre- and postmultiplication by a permutation matrix 


As you would expect, if we postmultiply a matrix by a permutation 
matrix, and premultiply it by another permutation matrix, we get a product 
in which we interchange the columns according to the right permutation 
matrix and rows according to the left permutation matrix. This is illus- 


trated in Fig. 10.16.3. 


0 OQ L Ol & 4 010 514 

g 10 Os 238 00 tis S 292 

0 0.0 LIT 4 5 100 2 80 5 

to DILO & 2 1 4*4 
Fic. 10.16.3 


The product moment of a permutation matrix 
The product of а permutation matrix by its transpose is the identity 


matrix. This is indicated as 
(10.16.1) 


The rule is illustrated by Fig. 10.16.4. 


000 
oo "0 ü 0 0 1 1 
o lw. (3b 10770 = 0 L0 0 
000 Tiit 000 0010 
1 0 0 0 B wg iB 0 0 0 1 
T т! І 
Fie. 10.16.4 


r the permutation matrix is pre- or postmultiplied 


It hethe A indi i 
does not matter w t the identity matrix as indicated in 


by its transpose; we still ge 
(10.16.2) 


тт = І 


Using the same example as in Fig. 10.16.6, we get Fig. 10.16.5. 
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on 8 LIO D I1 9 Lf g uU 
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Fic. 10.16.5 


Figure 10.16.5 is the same as Fig. 10.16.4 except that the transpose precedes 
the matrix. Although from Figs. 10.16.4 and 10.16.5 you can probably see 
why the permutation matrix pre- or postmultiplied by its transpose must 
equal the identity matrix, the reason for the rule may be clearer if we 
consider each row or column vector of the permutation matrix as an ei 
vector. The product moment of а matrix is simply one whose diagonal 
elements are the scalar products of a vector by itself, and whose non- 
diagonal elements are scalar products of one vector by another. The scalar 
product of an e; vector by itself, as you know, is 1, whereas the scalar 
product of an e; vector by another e; vector is equal to 0. 

It should be clear then that when we rearrange the order of the entities 
or the attributes in a data matrix, we are pre- or postmultiplying the matrix 
by а permutation matrix. You should also be able to determine under what 


conditions multiplication of a matrix by a permutation matrix does not 
alter one of its product moments. 


Multiplication of the unit vector by a permutation matrix 


If a permutation matrix is pre- or postmultiplied by a unit vector, the 
result is the unit vector itself. This is stated in Eq. (10.16.3) for a column 
vector and Eq. (10.16.4) for a row vector. 


wl 


1 (10.16.3) 
Vr = 1 (10.16.4) 


While these rules should be obvious because multiplication by the permuta- 
tion matrix does no more than interchange the elements of the unit vector; 
they are important for some kinds of statistical analysis. Therefore, We 


shall give a numerical example for both row and column vectors in Fig- 
10.16.6. 


оноо 
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моо о 
Bee ee 
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Fic. 10.16.6 
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10.17 Multiplication by an e;; Matrix 


Any product involving one or more e;; matrices may always be simplified 
by expressing the e;; matrix as the product of an e; column vector post- 


multiplied by an e; row vector. 


Postmultiplication by an e;; matrix 
The product of a matrix a postmultiplied by an e;; matrix may be 
written as 
ае;; = aei; (10.17.1) 
But we know that a matrix postmultiplied by an e; vector is simply the ith 
column vector of the matrix, as in 
ae; = a.i (10.17.2) 


Substituting the right side of Eq. (10.17.2) in the right side of Eq. (10.17.1) 


gives 
ае; = аё; (10.17.3) 


From this we see that a matrix a postmultiplied by an ei; matrix gives the 
ith column vector of a postmultiplied by an e; row vector. Another way of 
stating this rule is that a matrix a postmultiplied by an e;; matrix results in 
a matrix all of whose columns are 0 except the jth which is the 7th column 
ofa. Anumerical example is given in Fig. 10.17.1. 


3.42 4 0040 
0000 
4.2 1 m E [0020 
зог а- рота- о озо 
; 2 2 2 B 0 2 0 
a ез a. ез ае23 
Fig. 10.17.1 


Premultiplication by an e;; matrix 


We can also write the product of а matrix а premultiplied by an ei; 


matrix as 
еда = ее;а (10.17.4) 


But we know that а matrix premultiplied by an е; row vector gives the jth 
row vector of the matrix, so that 

ёа = aj. (10.17.5) 
Substituting the right side of Eq. (10.17.5) in the right side of Eq. (10.17.4) 


gives 
еца = ей, (10.17.6) 
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From Eq. (10.17.6) we see that а matrix a premultiplied by an e;; matrix 
gives the jth row of a premultiplied by a column e; vector. To put it 
another way, a matrix a premultiplied by an e;; matrix gives a matrix all of 
whose rows are 0 except the ith which is the jth row of a. A numerical 
example of this rule is given in Fig. 10.17.2. 


0000 А 7 : 0 ооо 
ü o 1 9 6 4 3/7 1116 4 3]=]6 4 3 
000 0 722 ооо 
езд а ГА а; еза, 
Ес. 10.17.2 


Pre- and postmultiplication by an e;; matrix 


A matrix may be premultiplied by an е:; matrix and postmultiplied by 
ап ep matrix, that is, one in which the element in the pth row and jth 
column is 1. The product may be written as 


€ijl65, = eie ;aese,, (10.17.7) 

But we know that a matrix premultiplied by an e; vector and postmultiplied 
by an e, vector gives simply the jpth element of a, so that 

ep = aj, (10.17.8) 


Substituting the right side of Eq. (10.17.8) into the right side of Eq. 
(10.17.7) gives 


€ijü65 = е;азе, (10.17.9) 
But the right side of Eq. (10.17.9) can be written as 
Eilge, = Ajplig (10.17.10) 


If we substitute the right side of Eq. (10.17.10) into the right side of Eq. 
(10.17.9), we get 


е:34ра = 0:4 (10.17.11) 
We see therefore that if a matrix a is premultiplied by an e;; matrix and 


postmultiplied by an ej, matrix the product is an e;, matrix multiplied by 


the jpth element of a. A numerical example of this rule is given in Fig. 
10.17.3. 


ооо о 
NOD њ фо 
& оњ моњ 


Fra. 10.17.3 
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The product of two e;; matrices 
If the matrix a in Eq. (10.17.11) is a unit matrix, and if j is equal to p, 
the product is e; This we can show as, first, 


е еа = Ajpliq (10.17.12) 


Then since the element from the jth row and jth column of the identity 
matrix is simply 1, Eq. (10.17.12) may be written as 

гё = Cia (10.17.13) 
Now suppose that j is different from p. The element in the jth row and 
pth column of the identity matrix is 0 unless j and р are equal; therefore, 
for j and p not equal, we have 

6;j£pa = 0 (10.17.14) 
In other words, an e;; matrix multiplied by an ep, matrix is 0 unless j and p 
are equal. Ап example of Eq. (10.17.13) is given in Fig. 10.17.4. 


000][00 0 000 
0:0 L|[0 0 0 |-]|0.L.9 
оо 0][0 1 0 000 


€23 €32 C22 
Fig. 10.17.4 


An example of Eq. (10.17.14) is given in Fig. 10.17.5. 


оо O0jf0 0 0 000 

00 10 01| = 0 0 0 

0 0 ооо 0 0 0 O 
0 


ез €23 
Fic. 10.17.5 


10.18 Powers of Matrices 


To raise matrices to powers, we simply multiply the matrix by itself a 
given number of times. This is the same as in scalar algebra. We shall here 


consider raising matrices only to those powers that are positive whole 


numbers. Е n 
Only square matrices can be raised to powers. To raise a matrix to the 


Second power we multiply it by itself once as in Fig. 10.18.1. 
$3 e Im ze 11 16 
To soul IS А У И З 8 27 
а? а а b 
Fie. 10.18.1 
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Figure 10.18.1 shows why a matrix must be square if we want to raise it to 
a power. The prefactor must have the same number of columns as the 
postfactor has rows; therefore, since it is the same matrix, the matrix must 
have the same number of columns as it has rows. The rules of exponents 


hold in the case of matrix algebra just as they do in scalar algebra. These 
rules we review in 


ata” = ат} (10.18.1) 
lar} = агу (10.18.2) 
Equation (10.18.1) says that if we multiply a matrix a raised to the power = 
by the same matrix raised to the power y, we get a matrix raised to the 
power (z + y). Equation (10.18.2) states that if we raise a matrix a to the 


power z and then raise the resulting matrix to the power y, we get the same 


matrix as if we raised a to the power ry. Examples using numbers for 
exponents are given in Fig. 10.18.2. 


aa? = aê 
[02° = ав 


Fic. 10.18.2 
SUMMARY 


1. The product of a matrix and its transpose: 
а. When the natural order of a matrix is premultiplied by its transpose, 
the product is called the minor product moment of the matrix. 
b. The minor product moment of a matrix is symmetrical; that is, 
(a'a) = a'a 
c. When the natural order of a matrix is postmultiplied by its trans- 
pose, the product is called the major product moment of the matrix. 
d. The major product moment of a matrix is symmetrical; that is, 
(aa^)! = аа! 
2. The major and minor product moments of matrices are not in general 
equal, that is, 
aa’ = a'a 
3. The product of a symmetric matrix pre- and postmultiplied by a matrix 
and its transpose is a symmetric matrix. If 


8 = 5 
Шеп 


(а'ѕа)! = 


I 
е 
© 
Q 


and 
(asa’)’ 25 


I 
8 
8 
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4. Square products of matrices: 
8. If z and y are vertical matrices, and z'y exists and is a square 


matrix, then zy’ also exists and is square. 


b. If z'y and xy’ exist, x and y are of the same order. 
c. The products z'y and y'z are called minor products and the products 


ay’ and yz’ are called major products. 


5. The trace of a matrix: 
a. The sum of the diagonal elements of a matrix is called the trace of a 


e 


matrix and indicated as 


ira = È ai 


. The trace of a matrix is equal to the trace of its transpose: 


ira — та 
The trace of the sum of matrices is the sum of their traces: 


ir [a +b] =tra+irb 


. The trace of the minor product of two matrices is equal to the trace 


of their major product moment: 


ir z'y = tray! = тух = tr ye’ 


. The trace of a major product moment is equal to the trace of a 


minor product moment: 
ir aa! = tr a'a 


The trace of a product of two matrices is the sum of products of 


corresponding elements of the two matrices. 
The trace of the product moment of a matrix is the sum of squares 


of its elements. 


Products involving vectors: 


a. 


Premultiplication of a matrix by a vector. К f 
(1) The vector must be a row whose order is the height of the 


matrix. н j 
(2) The product is a row vector whose order is the width of the 


matrix. 


. Postmultiplication of a matrix by a vector. 


(1) The vector must be a column whose order is the width of the 


matrix. E : 
(2) The product is a column whose order is the height of the matrix. 


. Pre- and postmultiplication by a vector. 


(1) The prefactor must be a row and the postfactor a column. 
(2) The triple product is a scalar quantity. 
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7. Multiplication of a matrix by a unit vector: 


a. Premultiplication of a matrix by a unit vector gives a row vector 


whose elements are the sums of corresponding column elements of 
the matrix. 


. Postmultiplication of a matrix by a unit vector gives a column 


vector whose elements are the sums of corresponding row elements 
of the matrix. 


. Pre- and postmultiplication of a matrix by a unit vector is a scalar 


that is the sum of all elements in the matrix. 


8. Multiplication of a matrix by an e; vector: 


a. Premultiplication of a matrix by an e; vector gives the ith row 


vector of the matrix: 


u , 
е;а = а; 


b. Postmultiplication of a matrix by an e; vector gives the 7th column 


vector of the matrix: 


ае; = а., 


c. Pre- and postmultiplication of а matrix by an е; and е; vector, 


respectively, gives the ijth elements of the matrix: 


u 
eae; = aij 


9. Products involving diagonal matrices: 


a. Ifa matrix is premultiplied by a diagonal matrix, each element in the 


ith row of the product is the corresponding element from the matrix 


multiplied by the ith element of the diagonal matrix. 
If 


Da=c 
then 


cy = Р;а;; 


. If a matrix is postmultiplied by a diagonal matrix each element in 


the jth column of the product is the corresponding element from the 


matrix multiplied by the jth element of the diagonal matrix. 
If 


ad =G 
then 


Gi; = aijd; 


. If a matrix is pre- and postmultiplied by a diagonal matrix, the ijth 


element of the product is the corresponding element from the 
matrix multiplied by the ith diagonal element of the prefactor ат! 
the jth diagonal element of the postfactor. 

If 


Dad = M 
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then 
М; = DiMijj 
10. Products in which: а] factors are diagonal: 


a. The product of any number of diagonal matrices is a diagonal matrix 
whose ith element is the product of the ith elements of the matrices. 


If 
D,DiD. = De 


then 
dv аф: 


b. All diagonal matrices of the same order are commutative one with 


another: 
р„рьр, = р.р, = DeDaD» ete. 
11. Powers of diagonal matrices: 
a. Any power of à diagonal matrix with no vanishing diagonal elements 
ned by raising each diagonal element to the power. 


is obtai 
If 

DF = ô 
then 

ô: = Di 


f a diagonal matrix with no vanishing diagonal 


b. The inverse 0 
he reciprocal of each diagonal 


elements is obtained by taking t 


element. 
If 
р = 4 
then 
1 
di = D. 


y diagonal matrix is obtained by taking the 


c. The square root of an. 
gonal element. 


square root of each dia 
If 
D = d 


d; = ND; 


12. Matrix products involving scalar quantities: 
а. If a matrix is pre- or postmultiplied by a scalar matrix each element 


of the matrix is multiplied by the scalar diagonal constant. 
b. A scalar matrix is commutative only with square matrices. 


then 
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13. Products involving scalar quantities: 
a. Multiplication of any matrix by a scalar quantity consists of multi- 
plying each element of the matrix by the scalar. If g is a scalar and 
b a matrix, and if 


gb =f 
then 


fij = gbi 


b. All scalar quantities commute with all matrices and with one 
another. If g and k are scalars and х, y, and z matrices conformable 
for multiplication, then 


gkxyz = xgykz = xykgz ete. 


14. Multiplication by the identity matrix: 


a. Multiplication of a matrix by an identity matrix leaves the matrix 
unchanged. 


Іа = а =a 


b. Identity matrices commute only with square matrices. 
c. Any power of an identity matrix is the identity matrix, that is, 


Pst 
15. Multiplication by the sign matrix: 


a. Premultiplication of a matrix by a sign matrix changes the signs of 
the elements in the rows corresponding to the — 1's. 


b. Postmultiplication by a sign matrix changes the signs of the ele- 
ments in the columns corresponding to the —1’s, 

c. Pre- and postmultiplication of a square matrix by the same sign 
matrix leaves the principal diagonal unchanged. 

d. Powers of a sign matrix: 


(1) A sign matrix raised to a positive or negative odd power yields 
the sign matrix. If k is odd, 


hath ai 


(2) A sign matrix raised to a positive or negative even power is the 
identity matrix: 


06 = 4% = р 


16. Multiplication by а permutation matrix: 


а. If à matrix is postmultiplied by а permutation matrix its columns 
are interchanged as the columns of an identity matrix were inter- 
changed to obtain the permutation matrix. 


b. If а matrix is premultiplied by a permutation matrix its rows are 
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interchanged as the rows of an identity matrix were interchanged to 


obtain the permutation matrix. 
c. Any permutation matrix pre- or postmultiplied by its transpose 


gives the identity matrix, that is, 
т'т = тт’ = І 
d. Апу permutation matrix pre- or postmultiplied by a unit vector 
yields the same unit vector, that is, 


ll 
m= 


lr21' and rl 


17. Multiplication by an e;; matrix: 
a. A matrix postmultiplied by an e;; matrix gives the ith column vector 


of the matrix postmultiplied by an e; vector: 
ае; = а.е; 
b. A matrix premultiplied by an ei; matrix gives the jth row of the 
matrix premultiplied by an e; vector: 
еа = ег, 
A matrix pre- and postmultiplied by e;; and epg matrices, respective- 
ly, yield the product of the ipth element of the matrix and the ei, 
matrix: 
€ijü6pq = Ajplig 
d. The product of an e;; by an e; matrix is an е matrix: 
C5 ;€ iq = Cig 
e. The product of an e;; by an ера matrix is 0 for 7 ғ p: 
её = 0 
19. The powers of a matrix: 
a. A square matrix may be raise 


number by multiplying it by itself the n 


by the exponent. 
b. The laws of exponents for sca! 


ага” = arty 


d to any power that is & positive whole 
umber of times indicated 


lars hold for square matrices: 


[а=] = gy 
EXERCISES 
1. Given 


k = ant? + ant} +... Annta + 2antite + 20132173 +... + 20а + 
2азуләхз + . . . 20007 +... + 2a(n p nZ[n i]. 
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(a) Define matrix and vector factors in terms of which k can be expressed. 


(b) Express k in terms of these factors. 


п 
2. Given the n by m matrix z. In the following summations, Dx, means “the sum 


t= 


of the elements from 1 to n in the jth column of z; Dr; means the sum of the 
{=1 


{= 
elements from 1 to m in the ith row of z. Show how each of the summations can 
be expressed in terms of z, 1, and e, or е; vectors. Be sure to indicate the order 


of computation. 


(а) 5m; b) Уа; (0 2227 
іп j=l Jzli-l 


For each of the following exercises, two matrices, 


(d) r2 


i=1J=1 


zand y, are given. Try to find 


as simply as possible two other factors, a and b, such that a X b = y. If one of 
these is the identity, indicate this merely by J with the appropriate subscript. 


3. 241 
{= z2 -[9 2 7 
578 y-[ ] 
L2 4 
4. 24 1 4 5 1 
132 7 2 5 
ele % g| PR ly edP 
124 2 2 
6. 3211 6 #28 3 
*-|4 832359 3 у=|4 32 8 
252 4 6 15 6 12 
7; 852-11 9 4 3 $9 
те 48 2 3 y=|12 6 26 
25 2 4 6: 10. 28 


3.42 0-9 ud 
 |4 8 6 _|8 10 1 
"s ora |а 25 

421 8 4 о 
9. 426 
sca ed EN: 
pi E 

Bein 000 

10. 532 0003 

214 _|[@ 001 
dM EAE дуд 

532 00023 


ANAN 


= Фоо о 


аге both matrices. 


п. 


12. 


13. 


14. 


15. 


16. 


17. 


18. 
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435 4 -8 5 
вар „52 53 
432 4 =3 9 
510 b c 
251 9) REI 
ML S P A МЕ 
2458 -2 -4 -8 
572 Ba Wh @ 
235 523 
LET MT LEA. 
zy zd 
3258 832 
473 26 5 
ELA 473 
zoe dios yo e 
312 Bd 4 


What is the relationship between 
(a) the major product moments of z and y in Ex. (11) and (13)? 


(b) the minor produet moments of z and y in Ex. (12) and (14)? 


Let 
5 0 0 
40 0 1 
х=|0 9 0 z = 03 0 
0 0 25 1 
0 0 5 
What is k? 
Let 
8 2 4 1 
23 3 
= l=] —1 
ela 88 i 
31 2 
What is 21? 


Given a matrix z, let a = 711’. Assume a square. Express very simply (a) in 
symbols, (b) in words — the trace of a as a function of the elements of z 


ANSWERS 


(a) a is an nth-order symmetric matrix; x is an nth-order vector 


(b) k = тах 


10. 


п. 


13. 


SIMPLE MATRIX COMPUTATIONS 


. (a) Vie; (b) [eiz]l (c) [I'z]l (d) l'[z1] 


а=0 100] ф=% 
а = І, ъ= [0 1 0] 


1 200 
a=(0 100) = [о 6.a-|0 1 0 
0 003 
30020 
0200 
ede ал: 
0002 
2000 à 
0200 
8 0) Ду 2.0 ъ= І, а = І, “со 
0002 
0 
a=(1/0 0 300 $92 
0 
0 
a-h Ъ=|1|[0 0 o 1 
0 
Коб 12. -1 0 
а= b=|0 -1 0 dm od 
Or. On 00 
0 0 
& qq 14. 0010 
а=. b-|oo: oe oe 02g 
1 9:8 0001 
0100 
- (a) zz! = уу (Ы) z/z = уу 
. k = —1/2 
5 
2 
“= 
4 


„ (а) ira = rri 
(а) 


(b) The trace of a is the sum of the elements in z 


ono 


b=], 
0 
0 
2 
0 
0 
0 
1 
b=]; 


Chapter 11 


Multíplícatíon of Supermatríces 


In the behavioral sciences it is often desirable to partition data matrices 
by entities or attributes or both. A particular study may logically call for 
subsets of entities and attributes. For example, in a matrix of test scores, 
grades, ratings, and physiological and other measurements on a large group 
of military service personnel, it may be helpful to partition the entities ac- 
cording to service schools attended, and the attributes according to type of 
measure, for example, test scores and grades. It is also convenient some- 
times to partition entities or attributes or both for computational con- 
venience. It may be that the computational facilities available will not 
handle the complete data matrix as a whole or matrices derived from it. It 
is therefore desirable to have a systematic procedure for handling the data 
in subsets or appropriate combinations of submatrices. 

The addition and subtraction of supermatrices cause little difficulty either 
in principle or application. The multiplication of supermatrices is simple in 
principle, but in practical applications even competent mathematicians are 
easily confused. In typical textbooks on matrices, the principles of super- 
matrix multiplication are briefly explained but no adequate notation is pro- 
vided to help the student or research worker avoid confusion. 


11.1 A Notation for the Multiplication of Supermatrices 


Partitioning vectors 

To understand how supermatrices are multiplied we must first develop a 
You recall that a supermatrix may be obtained by merely 
partitioning a simple matrix between certain rows and columns. Let us 
begin with a simple matrix X of P rows and Q columns. But, instead of 
writing the matrix as Хро to show that it has P rows and Q columns, let us 
write it as „Хо. Here we call P a prescript and Q a postscript. The prescript 
tells us how many rows the matrix has and the postscript tells us how many 


notation. 


columns it has. 
Next let us define what we shall call partitioning vectors. A prepartitioning 
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veclor or, as we shall sometimes call it, а prepartitioner, indicates how we 
shall partition the matrix by rows, and a postpartitioner indicates how we 
shall partition it by columns. A prepartitioner is a column vector whose 
order is equal to the number of matric rows in the partitioned matrix. Each 
element in the prepartitioner is a scalar indicating the number of simple 
rows of the matric elements in the corresponding matric row. A post- 
partitioner is a row vector whose order is equal to the number of matric 
columns of the partitioned matrix. Each element in the postpartitioner is a 
scalar indicating the number of simple columns of the matric elements in 
the corresponding matric column. 

You see, therefore, that a prepartitioner is always a column vector and а 
postpartitioner is always a row vector. You see also that the sum of the 
elements in the prepartitioner must always be equal to the number of simple 
rows in the matrix, and the sum of the elements in the postpartitioner must 
be equal to the number of simple columns in the matrix. 


Prepartitioning 
Let us consider now the prepartitioning vector in Eq. (11.1.1). 


а 


а 


la) = (11.1.1) 


a, 
Notice that a, on the left of Eq. (11.1.1) is enclosed in brackets. This is to 
show that it is a vector rather than the rth element of a vector, The sub- 


script r in the symbol [a,] means that the vector has r elements. If now we 


want to show that the matrix „Хе has been prepartitioned by the vector [ar] 
we would write it as 


tal [РХо] = (Xo (11.1.2) 


The column vector prescript [0,] on the right side of this equation means 
that Хо has been prepartitioned into r matric rows. We now have a type 
III column vector. You will recall that a type III vector is a vector whose 
elements are matrices. According to Eq. (11.1.1), the first matric element 
has a; simple rows, the second matric element has а» simple rows, and so on. 
Since the sum of the elements in [a,] must be equal to P, we can write 


1 [a] = P (11.1.3) 


where, of course, [1,]’ is a row unit vector of order r. 


To illustrate Eq. (11.1.2), in „Хе we let P = 5 and Q = 4. We then take 
the numerical example in Fig. 11.1.1. 


MULTIPLICATION OF SUPERMATRICES 219 


123 5 
43 7 6 
pXg=ska=/4 2 1 6 
8 6 9 5 
43 2 7 

Fig. 11.1.1 


Now suppose we wish to prepartition 5X, into two matric rows so that the 
first matric element has 3 simple rows and the second matric element has 2 
simple rows. The prepartitioner would then be as given in Fig. 11.1.2. 


_ ET 
[ar] = Н 
Fic. 11.1.2 


As indicated here, when we write out the numerical elements of a parti- 
tioning vector we shall frequently attach a subscript to each element to 
indicate its position in the vector. Using the example in Fig. 11.1.1, we can 
then illustrate Eq. (11.1.2) by Fig. 11.1.3. 


sao gru 


8695 
4327 
Га. 11.1.8 
Postpartitioning 
Let us next consider the postpartitioning vector in the equation 
[b] = [bj b» ... ba) (11.1.4) 


Notice that the left side of Eq. (11.1.4) is primed to show that we havearow 
t as in ordinary vector notation. The subscript s in the symbol 
ments. If now we want to show that the 
d by the vector [b.]" we write it as 


vector, jus 
[b,]' means that the vector has s ele 


matrix „Хо has been postpartitione: 


[pXo] u^ = PX wu (11.1.5) 


The row vector postscript on the right side of Eq. (11.1.5) means that „Хе 
has been postpartitioned into s matric columns. This then gives us a type 
III row vector. According to Eq. (11.1.4), the first matric element has b; 
simple columns, the second matric element has b; simple columns, and so on. 
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Since the sum of the elements in [b;]" must be equal to Q, we can write 


0.7 [1.] = Q (11.1.6) 


where 1, is a unit vector of order s. 

Let us assume now that we wish to postpartition the matrix 4X; in Fig. 
11.1.1 into two matric columns so that both the first and second matric ele- 
ments have two simple columns. The postpartitioner would then be 
given as 


[b.]’ = [2:27] 


Fig. 11.1.4 


Then using the numerical example of Fig. 11.1.1 to illustrate Eq. (11.1.5), 
we have Fig. 11.1.5. 


Ll 2 9: 5 
4 3 7. с) 
PX tbi" = 5X [2,25] =|4 2 k S 
8 6 9.5 
& 3 2 T 
Fig. 11.1.5 


Pre- and postpartitioning 


Suppose next we wish to show that the matrix „Хо has been both pre- 
partitioned by [a,] and postpartitioned by [b,]'. This we would indicate by 
tad [eX o] ы, = tX ta (11.1.7) 


To illustrate this let us again take >ьХо as given in Fig. 11.1.1 and assume 
that the prepartitioner is as given in Fig. 11.1.2and the postpartitioner as in 
Fig. 11.1.4. This would result in the supermatrix given in Fig. 11.1.6. 


- 
to 
w 


5 


ta X tyr = [2] X oa = 
à 


Fie. 11.1.6 


As indicated by the numerical examples in Figs. 11.1.3, 11.1.5, and 11.1.6, 
we can actually write out the elements of the vector subscripts when their 
numerical values are specified. However, we can also write these vector 
subscripts in expanded notation, even when the numerical values are not 
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given. For example, we can indicate a prepartitioning of Хо as in Eq. 
(11.1.8). 
(11.1.8) 


ar 


«1Хо = Hi 
a: 
We can also indicate a postpartitioning as 
Хы, = PX, b ш. м (11.1.9) 


То show both a pre- and postpartitioning in expanded subvector nota- 
tion we would have Eq. (11.1.10). 


ow - B cs (11.1.10) 


ar 


Expanded notation for the type Ill vector 


Remembering now that Eq. (11.1.8) is a type III column vector, let us 
see how the right side may be rewritten to show the individual matric ele- 


ments of the vector. This we do in Eq. (11.1.11). 
Хо 
«Хе = «Хе (11.1.11) 
Хо 
On the right side of Eq. (11.1.11) we now have given the matric elements of 
the type III column vector. The prescript gives the number of simple rows 
of the element and the postscript the number of simple columns. Since 
there is only one matrie column and since all matric elements within a 
column must have the same number of simple columns, we have the same 
postscript, Q, for each element. In an actual case, each element on the right 
of Eq. (11.1.11) might be a matrix of all measures available on the military 


service personnel in а particular technical training school. 


To express the type III row vector in terms of its matric elements, we 


write the right side of Eq. (11.1.9) as 

pXp, = [РХь PXo --- Хь] 
We now see that the prescript Р іп the right side of this equation indicates 
the number of simple rows of the matric elements, and the postscripts indi- 
cate the number of simple columns. Again, in an.actual case, each element 


on the right of Eq. (11.1.12) might be a matrix of particular measures, such 


as test scores on all service personnel in all technical training schools. 


(11.1.12) 


Expanded notation for the supermatrix 


To express the supermatrix in terms of its matric elements, we write the 


right side of Eq. (11.1.10) as in Eq. (11.1.13). 


222 SIMPLE MATRIX COMPUTATIONS 


m aX. eap 
юлы, = «Хы e ... а (11.1.13) 
аХь aX cos a Xa, 


Again the prescripts in the right side of Eq. (11.1.13) indicate the number 
of simple rows of the submatrix, and the postscripts indicate the number of 
simple columns. You will note that all submatrices within a given matric 
row have the same prescript, and all submatrices within a given matric 
column have the same postscript. This must be 80, of course, because of the 
way supermatrices are formed from simple matrices. Here each element on 
the right of Eq. (11.1.13) might be a matrix of a particular type of measures 
on personnel in a particular school. 

We may use the numerical example of Fig. 11.1.3 to illustrate the sub- 
matrices on the right of Eq. (11.1.11). This we do in Fig. 11.1.7. 


l2 3-5 

aXs=]4 3 7 6 

4215 

„_ [8 805 

х, = [8 89 4 
Fig. 11.1.7 


To illustrate the submatrices on the right of Eq. (11.1.12), we use the 
numerical example in Fig. 11.1.5, This gives us Fig. 11.1.8. 


: 2 3 5 
4 3 7: 16 
sXe, = |4 2 Xa = |1 5 
8 6 9 5 
4 3 D" 
Fic. 11.1.8 


Figure 11.1.9 uses the numerical example in Fig. 11.1.6 to illustrate the sub- 
matrices on the right of Eq. (11.1.13) 


2 3 5 
3 aX4 =|7 6 
2 ї 5 


6 9 5 
d X= |9 4 


Fie. 11.1.9 


aX a 


Хь = [ 


Ф оо fF Re 
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Simplified notation for type Ill vectors 


А Next let us see how we may introduce a slight simplification into the nota- 
tion of Eq. (11.1.11). We can express the right side as in Eq. (11.1.14). 


PD d 
ш Хо = | (11.1.14) 
aX Jo 
Notice that on the right of Eq. (11.1.14) we have dropped the postscript Q 


from each of the matric elements and written it as a postscript outside the 
brackets. This means that Q belongs as a postscript to each of the elements 


inside the brackets. 
We can also simplify the right side of Eq. (11.1.12) by writing it as 


PXuy = Хы Хь ... Хы] (11.1.15) 


On the right of Eq. (11.1.15) the P as a prescript to the brackets means that 
it belongs as a prescript to each of the matric elements. 

In Eq. (11.1.14) the scalar subscript Q is a postscript; therefore we know 
from the left side of Eq. (11.1.14) alone that we have a type III column 
vector. If the scalar subscript is a prescript, we have a type III row vector. 
In Eq. (11.1.15) the scalar subscript P is a prescript, therefore we know 
from the left side of Eq. (11.1.15) alone that we have a type III row vector. 


11.2 The Type IV Supervector 
The supermatrix as a column supervector with type Ill row vector elements 


We shall now see how we may express a supermatrix as a type of super- 
vector. You recall that we expressed a simple matrix as a type II super- 
vector whose elements were simple vectors. We begin by writing the matric 
row vectors on the right of Eq. (11.1.13) as in Eq. (11.2.1). 


„Хъл = Хы «Хы ... Xd) 
“Хы = [Xs Хь oee =Хь] (11.2.1) 
a X (^ = [„Хь, Хь, qoe „Хь] 


But we have already seen that the prescripts for the elements of a type III 
row vector may be taken as а prescript outside the brackets, so that Eq. 


(11.2.1) may be written as in Eq. (11.2.2). 


aXmr = [Хы Xs, ... X 
Pr xax ad 
„Хы = [Хы Xr, ы] € 


„Хы = [Хх Xn v ЖАЫ] 
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Then because of Eqs. (11.2.1) and (11.2.2) we may express the supermatrix 
la) X p, аз a column vector whose elements are type III row vectors. This 
we do in Eq. (11.2.3). 


a X tar 
“Хь, = «Жы, (11.2.3) 
a X t) 


But since the vector postscript is the same for each element in the right of 
Eq. (11.2.3) we may write 


aX 
[iX] = «t (11.2.4) 
x UL 
Аз you know by this time, the vector postscript [b,]’ outside the brackets on 
the right of Eq. (11.2.4) means that it belongs as a. postscript to each ele- 


ment inside the brackets. 
The supermatrix as a row supervector with type Ill column vector elements 


We may also express the type III column vectors of the supermatrix on 
the right of Eq. (11.1.13) by means of Eq. (11.2.5). 


aX aX aX 
(P d & a 

to Xo = | ta] X, = vi ta] Xo, = A 
„Х bı aX be aX b, 


a 1.2.5) 


Because of Eq. (11.2.5), therefore, we may express the supermatrix 


lad Xto aS a row supervector whose elements are type III column vectors. 
This we do in Eq. (11.2.6). 


„(Хь = „Хь Хы... X4] (11.2.6) 


The vector prescript [a,] outside the brackets on the right of Eq. (11.2.6) 
means, of course, that it belongs as a prescript to each element within 
the brackets. 

We shall now designate supervectors of the type given by Eqs. (11.24) 
and (11.2.6) as type IV supervectors. A type IV column vector is a column 
supervector whose elements are type III row vectors. А type IV row 
vector is a row supervector whose elements are type III column vectors. 

Notice that we can tell from the left-hand sides of Eqs. (11.2.4) and 
(11.2.6) whether we have a column or a row type IV vector. If the post- 
partitioner is outside the brackets, we have a column type IV vector as in 
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Eq. (11.2.4). If the prepartitioner is outside the brackets, we have a type IV 


row vector, as in Eq. (11.2.6). 
We may illustrate the type IV column vector of Eq. (11.2.4) by using the 


numerical example of Fig. 11.1.1 in Fig. 11.2.1. 


lian Xle = [eX]. TE 


Fia. 11.2.1 
To illustrate the type IV row vector of Eq. (11.2.6), we use the same 


numerical example in Fig. 11.2.2. 
11, = [je zi) = 


Fic. 11.2.2 


11.3 The Transpose Notation for the Supermatrix 


Transpose of the type ll column vector 
Next let us see how we indicate the transpose of a supermatrix by means 
of the notation we have just developed. We shall begin with the type III 
column vector given by Eq. (11.1.11). To take the transpose of the left side 
le very similar to that for the transpose of a 


of Eq. (11.1.11) we use à ru 
product of simple matrices. Here you recall we placed the symbols for the 


matrices in reverse order and took their transposes. Similarly we place 
the three symbols on the left of Eq. (11.1.11) in reverse order and take the 


transpose of each symbol, thus 

[ia Хо}! = eX ton’ (11.3.1) 

You will notice on the right of Eq. (11.3.1) that we have not primed the 
rescript Q since it is a scalar quantity. 

ore nspose of the right side of Eq. (11.1.11), we first write the 


To take the trà ) 
column as a row, and indicate the transpose of each element just as we 
learned to do in taking the transpose of a supermatrix. This gives us 
Eq. (11.3.2). 

a Xo Н 

«Xo [Хо Хо) ... GXoYl (11.3.2) 


„Хо 
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'To take the transposes of the elements in the right of Eq. (11.3.2) we must 
put the symbols in reverse order and take their transposes. But since the 
subscripts are all scalars, we prime only the X symbol. Therefore we have 
Eq. (11.3.3). 


[Хо] = oX;, 
1.Хо] = oX;, (11.3.3) 
la Xal = oX7, 

Substituting the right side of Eq. (11.3.3) in the right of Eq. (11.3.2) gives 

Eq. (11.3.4). 


“Хо : 
"е ч. S ETT (113.4) 
„Хо 


But we can take the Subscript Q in the right of Eq. (11.3.4) as a prescript to 
the brackets, as in Eq. (11.3.5). 


“Хо i 
v БР ИЕ 2 йз) 
„Ха 


From Eqs. (11.1.11), (11.3.1), and (11.3.5), therefore, we may write 


(Хо) = Xt,» = XA XL o... x1] (11.3.6) 


Remember that on the right of Eq. (1 1.3.6) the Q belongs as a prescript to 
each of the elements. 

Let us now compare the right side of Eq. (11.3.6) with the right side of 
Eq. (11.1.14) to see what rules we have used to take the transpose of the 
column type III vector. There are four distinct rules: (1) the column 8 
written as a row, (2) the symbol X for the submatrices is primed, (3) the 
prescripts of the submatrices are Written as postscripts, and (4) the post- 
script Q to the brackets is written as a prescript. 


Transpose of a type Ill row vector 


We should now be able to write the transpose of а type III row vector 85 
given in Eq. (11.1.15). To take the transpose of the left side of Eq. (1 1.1.15) 
we reverse the order of the symbols and take their transposes. To take 
the transpose of the right side of Eq. (11.1.15) we use the following four 
rules: (1) The row is written as a column. (2) The symbol X for the 
submatrices is primed. (3) The postscripts of the submatrices are written 
as prescripts. (4) The prescript P to the brackets is written as a postscript- 
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We therefore get Eq. (11.3.7) from Eq. (11.1.15). 
WX! 
7 
Xer) = tX» = d (11.3.7) 
A’ |p 
Examples of transposes of type Ill vectors 


Let us see now what the transpose form of a type III column vector given 
by Eq. (11.3.6) means in terms of a concrete example. We have the column 
form given in Fig. 11.1.3. We can illustrate the transposed row form in 


Fig. 11.3.1. 


On Ооо 4 
OU e tue 
л © o оо 
^ Ob оо 


1 

2 

eX(ay = АХ з = 3 
5 

Fie. 11.3.1 


The submatrices of the row form are therefore as in Fig. 11.3.2. 


144 8 4 
232 6.3 
Ха = |а 7 1| "|o 2 
565 57 

Fic. 11.3.2 


It is clear that in simple form the right-hand side of Fig. 11.3.2 is simply 
the transpose of the right-hand side of Fig. 11.1.3. It is also clear that the 
left and right examples in Fig. 11.3.2 are simply the transposes of the upper 
and lower examples of Fig. 11.1.7. This is, of course, as it must be, and 
merely serves to illustrate concretely the notation we use in indicating the 
transpose of a type III column vector. | 

Next we shall see what the transpose form of a type III row vector given 
in Eq. (11.3.7) means in terms of a concrete example. We have the row 
form given in Fig. 11.1.5. We illustrate the transposed column form by 


Fig. 11.3.3. 
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The submatrices of the column form would be the transposes of the 
submatrices in Fig. 11.1.8 of the row form. These we show in Fig. 11.3.4. 


"REN pe E INT: 

die 3 26 5] 
ac yu qug) 

„= [3 6 5 5 4 
Fic. 11.3.4 


The transpose of a type IV supervector 


We can now show how we indicate the transpose of a type IV vector. 
First we shall consider the transpose of the type IV column vector in Eq. 
(11.2.4). The rules are very much the same as for indicating the transpose 
of a type III column vector. 

To take the transpose of the expression on the left of Eq. (11.2.4), We 
write the symbols in reverse order and take the transpose of each symbol. 
Since both subscripts are vectors, this means that the transpose of each 
must be indicated. We therefore write the transpose of the left side of 
Eq. (11.2.4) as Eq. 11.3.8. 


toa Xlor) = walX ían] (11.3.8) 


The transpose of the right side of Eq. (11.2.4) is taken exactly as it is for e 
type III vector except that the row vector postscript [b,]’ must be unprime 


when it becomes a prescript. We have therefore Eq. (11.3.9) as the trans- 
pose of Eq. (11.2.4). 


(Хь) = [Хы] = ьа Xi... X] — 0139) 


Using the numerical example in Fig. 11.2.1, we may illustrate Eq. (11.8.9) 
by Fig. 11.3.5. 


144 8 4 
2 3 2 6 3 
ыы Хш] = Къ ы] = к= 
P o Т 9 2 
5 6 5 5 T7 
Fig. 11.3.5 


The type III column vector elements on the right side of Eq. (11.3.9) ие 
illustrated by Fig. 11.3.6. 


144 8 4 
? 3 3 6 3 
Xs = =|— 
ере m qu 
5 6 5 5-7 


Fie. 11.3.6 
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You see then that aside from the fact that the transposes of vector sub- 
scripts must be indicated, the rules for taking the transpose of the type IV 
column vector given by Eq. (11.2.4) are the same as those for taking the 
transpose of the type III column vector given by Eq. (11.1.14). This is 
also true for the transpose of the type IV row vector. We therefore write 
the transpose of the type IV row vector directly from Eq. (11.2.6) as in 
Eq. (11.3.10). 
WX. 

1 
tX (11.3.10) 
wX ion 


example in Fig. 11.2.2, Eq. (11.3.10) may be illustrated 


(Хъл) = naX" len = 


Using the numerical 
by Fig. 11.3.7. 


ES 
№ 
оҥ 

кюк № њ 
сл оо оо 
mE 
М 


n the right of Eq. (11.3.10) are illus- 


[ma X lan = [m* o " = | 7 
Fig. 11.3.7 


The type III row vector elements о 
trated by Fig. 11.3.8. 


p 4 4 8 4 

a Xt 22] | 3 Ø | 6 4 

8 T 1l 9 2 

Xin "IE 6 5 | 5 | 
Fre. 11.3.8 
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Notation for the minor product 
products of type IV vectors or supermatrices 
how we multiply type III vectors. As in 
multiplication of simple vectors, we have both a minor and major product. 
First we shall consider the minor product of two type III vectors. As you 
would suppose, the minor product of two type III vectors consists of a row 
vector postmultiplied by а column vector, just as in the case of simple 
vectors. Here the minor product of two type III vectors is the sum of 
products of corresponding elements. Therefore, it must be clear that for 
the minor product of two type III vectors to exist the vectors must have 
the same number of elements. Furthermore, since the products are formed 
from corresponding elements of the two vectors, the number of simple 


Before considering how the 
are formed we shall first see 


230 SIMPLE MATRIX COMPUTATIONS 


columns in a given element of the prefactor must be the same as the number 
of simple rows in the corresponding element of the postfactor. A concise 
way of combining these rules is as follows: for the minor product of two 
type III vectors to exist, the postpartitioner of the prefactor must be the 
transpose of the prepartitioner of the postfactor. This rule can be easily 
demonstrated. Suppose we have a column type III vector given by Eq. 
(11.4.1). 


ым = (11.4.1) 

A 
We wish to premultiply both sides of Eq. (11.4.1) by the corresponding sides 
of Eq. (11.1.15). This gives us the minor product indicated by Eq. (11.4.2). 


PX wv Yu = pP[Xo, Хь, ... Xu] (11.4.2) 


Ум 
Multiplying out the right side of Eq. (11.4.2) and attaching the external 
scalar subscripts P and M to the product terms, we have 


PX pyr Y = [PX Yar + PX Ум +... H РХ, Ум] 
(11.4.3) 
Or if we sum the products on the right of Eq. (11.4.3) to get a single matrix 
Z, we have 
PX bV лм = Р2м (11.4.4) 
Notice now on the left of Eq. (11.4.4) that the vector postscript [b,]/ of X is 
the transpose of the vector prescript [b] of Y. Notice also that the right 
side of Eq. (11.4.4) is a simple matrix with the same number of rows, P, 28 
the prefactor X and the same number of columns, M, as the postfactor Y. 
Furthermore, there is nothing to indicate on the right of Eq. (11.4.4) what 
were the adjacent subscripts or common superorder of the factors on the 
left. These rules of dimensionality are analogous to those for simple 
matrices. 
Numerical illustration 


We can illustrate Eqs. (11.4.3), (11.4.4), and (11.4.5) by Fig. 11.4.1. 


2 ud 
3 2 
2 3 4 $ 23] 
3X n згу —|1 4 1 1959 
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Fic. 11.4.1 (Cont. on next page) 
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13 8 16 4 29 22 58 34 
14 9/+] 4 1/+]14 11|2|32 21 
Tow 8 2 31 24 46 30 


Fic. 11.4.1 (Cont.) 


Figure 11.4.2 isa diagrammatic representation of the right-hand side of the 


Sg gag 
ЕЛЕЕ: 


Fra. 11.4.2 


1l 


\ 


the minor product 

To take the transpose of the minor product of type III vectors involves 

no new principles. On the left side of Eq. (11.4.3) we take the product of 

the transposes in reverse order, Equations (11.3.5) and (11.3.6) show how 
f a type III column vector, and Eq. (11.3.7) shows 


we take the transpose © 
how we take the transpose of a type III row vector. We therefore get 


Eq. (11.4.5) from (11.4.2). 


Transpose of 


[ьХтьџ’ to! Yul’ = MY toa tX» = al Yor You ЖУ Yi] 
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Multiplying out the right side of Eq. (11.4.5) and attaching the external 
scalar subscripts M and P to the product terms, we get 


мы ba Xp = [xY Xp + Ys, kp Facs t MY, Xp] 
(11.4.6) 


Notice that we could have obtained the right side of Eq. (11.4.6) directly 
by taking the transposes of each of the product terms on the right of 
Eq. (11.4.3). If we sum the products on the right of Eq. (11.4.6), we may 
write 


Yt (Xp = ma (114.7) 


Using the same numerical example as in Fig. 11.4.1, Eqs. (11.4.6) and 
(11.4.7) are illustrated in Fig. 11.4.3. 


31 

А [2.37212 4 514 
a Y 19 39 por E E Е 4 3 " HE 12] + f 2 Ji : 
3a a 
3 
2 


раат вав) рооза 31] Гэв: | 
8 94 412 22 11 24| < |34 21 30 
Fig. 11.4.3 


11.5 The Minor Product Moment of a Type Ill Vector 


We define the minor product moment of а type III vector in the same 
way asfor a simple vector. The minor product moment of a type III vector 
is a type III column vector premultiplied by its transpose. We must there- 
fore distinguish two kinds of minor products of type III vectors depending 
on whether the type III vector is a column vector or row vector. Remember 
that a matrix may be partitioned either into a type III column or row 
vector, whether the simple form is vertical or horizontal. Ordinarily We 
would think of the natural order of a simple data matrix as being vertical. 
However, a vertical simple matrix, as you know, may be partitioned into 
a row type III vector. Therefore, whether a type III vector is a row О? 
column tells us nothing about whether the simple form is vertical or hort- 
zontal. Therefore, even though with simple matrices and vectors the use 0 
the prime ordinarily indicates a horizontal form, with supermatrices and 
their elements it indicates only a transpose, 


Minor product moment of type Ill row vector 


First, then, let us consider the minor product moment of a type III row 
vector. Using Eqs. (11.1.15) and (11.3.7), we have 


PX toa ыХ> = БХ XQ ... Хь] XX (11.5.1) 
pA’ |p 
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Multiplying out the right side of Eq. (11.5.1) gives 


„Хь wXp = [Хь Xp + РХ, Xp +... + РХ, Хь) 
(11.5.2) 


Notice now that although Eq. (11.5.2) is a minor product moment, the 
primed factor is the postfactor on the left side of the equation and in each 
term on the right side also. This is different from the practice you are 
used to in the case of minor products of simple matrices and vectors, where 
the prefactor is customarily the primed factor. Note that each term on the 
right of Eq. (11.5.2) is a product moment of a simple matrix. It may, 
however, be either a major or minor product, depending on whether its 
common order b; is greater or less than the distinct order P. To illustrate 
Eq. (11.5.1) and (11.5.2), we can use the numerical example for X in 


Fig. 11.4.1 and get Fig. 11.5.1. 
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3441 
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Fig. 11.5.1 
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Figure 11.5.2 shows diagrammatically how the minor product moment of 


а type Ш vector is formed. ш " 
EBB EEE 


Ета. 11.5.2 
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You will note that the product terms in Figs. 11.5.1 and 11.5.2 are all 
major product moments except the last. 


Minor product moment of type lll column vector 


Next let us consider the minor product moment of a type III column 
vector. Suppose we take the minor product moment of the type III column 
vector given by Eq. (11.4.1). This would give us Eq. (11.5.3). 

aX 


мҮъ waYar = Ml Yo, Vise oe 24 (11.5.8) 


aY 1м 
Multiplying out the right side of Eq. (11.5.3) gives 
“Yoy [b,) Yu = [уь by Үм + MY on bY M duis мъ, »,Y м] 
(11.5.4) 


Using the numerical example for Y in Fig. 11.4.1, we illustrate Eq. (11.5.3) 
with Fig. 11.5.3. 
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If now you look at the right-hand sides of the equations in Figs. 11.5.1 
and 11.5.3, you have no way of knowing which is the minor product moment 
of а type III row vector and which is the minor product moment of а 
type III column vector. However, you can readily tell from the left-hand 
sides of the equations. In the former case the primed factor is the post- 
factor, while in the latter case it is the prefactor. In the same way you сап 
tell from both sides of the equation that Eq. (11.5.2) is the minor product 
moment of а row vector and (11.5.4) is the minor product moment of 2 
column vector. You can always tell that a product moment is a minor 
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product moment because adjacent subscripts are vectors (enclosed in brack- 
ets) rather than scalars. 
11.6 The Major Product of Type Ill Vectors 

Notation for the major product 

As in the case of simple vectors, the major product of type III vectors 
has a column vector for the prefactor and a row vector for the postfactor. 
The clement in the ith row and jth column is the ith element of the 
prefactor postmultiplied by the jth element of the postfactor. Equation 
(11.6.1) shows how we indicate the major product of two type III vectors. 


Pod 
ta) Xo oY ta! = ak ys. Ka кез. ЖУ] (11.6.1) 
„Хх 
Expanding the right side of Eq. (11.6.1) gives Eq. (11.6.2). 
„Хо оҮа aXooYa < „Жоон 
„ХооҮа aXo0Ya ·· „Хо Ve 
aaea m 96 Иб, “ste 
„Хо aa „Хо Ye e a XQ Yar ims 
The right side of Eq. (11.6.2) may be written more compactly аз 
aXXo oY tea’ = шал! (11.6.3) 


Now let us summarize the rules for the major product of type ш vectors 
as indicated in Eq. (11.6.3). First, we see from the left side of this с 
that the scalar postscript Q of the prefactor must be the same as the ies = 
Prescript of the postfactor. This merely means that the factors must be 


conformable, as they must be for simple matrices. Second, we see b 
Product on the right of the equation is a supermatrix that па ere : 
as the prefactor and postpartitioned as the postfactor. Finally, T. veh а 

the product tells us nothing about ће common dimension of the factors. 


Numerical illustration 


A numerical example of the major product of type III vectors is given in 


Fig. 11.6.1. 
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Fig. 11.6.1 (Cont.) 


Using the same orders as in Fig. 11.6.1 


, we illustrate the major product 
diagrammatically in Fig. 11.6.2. 
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Eq. (11.6.4). 
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" " " 

aYo Ха aYo Xas ees aYo Xa 

PF X ® 4^ d u ГА 

а | Ts uan ... fasi 
(А ^ io 

e Yo Xa «Yo Xas 96 «Yo Xa, 


(11.6.4) 

More compactly we could take the transpose of both sides of Eq. (11.6.3) 
to get 

led Yo «Хш, = {л (11.6.5) 

Figure 11.6.3 shows the transpose of the numerical example given in Fig. 
11.6.1. 
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Fig. 11.6.3 


11.7 The Major Product Moment of Type III Vectors 


The major product moment of a type Ill column vector 

As in the case of minor product moments of type III vectors, we also have 
two kinds of major product moments, namely, the major product moment 
rom a column vector and the major product moment from a row vector. 
Equation (11.7.1) indicates the major product moment of a column vector. 


aX 
ta) Xo eX ш, = «= dX, Хы... Xj Ser) 
„Х| 
Expanding the right side of Eq. (11.7.1) gives Eq. (11.7.2). 
a Xo oXa «Хо oX;, e «XQ Xa 
ta) Xo eX ш, = As Di wa des d А 
EO т ANS vr d 


(11.7.2) 
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A numerical example of а major product moment of a type III column 
vector is given by Fig. 11.7.1. 
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Fig. 11.7.1 


It should be clear from Fi 
III vector is a symmetri 
The major product mo 


Е. 11.7.1 that the major product moment of a type 
c simple matrix symmetrically partitioned. 
ment is illustrated diagrammatically by Fig. 11.7.2- 
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Fig. 11.7.2 


The major product moment of a type Ill row vector 


pe III row vector we would 


To indicate the major product moment of a ty э 
The major product 


Simply premultiply the row vector by its transpose. - 
moment of a type III row vector is given by Eq. (11.7.3). 


E 
ыХррХь ыХррРХь ... „Хр РХь, 
TE oXpPXn ьХррХь ... oXprX, 
aX p PX wr = у e T" Ж 
^ 
„ХьРХь ьХ>РХь ... ьХррХь, 


(11.7.3) 
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In Eq. (11.7.2), which represents the major product moment of а b os 
column Sector: the primed matrices are the postfactors on both sides of | е 
equation. In Eq. (11.7.3), which represents the major product moment of а 


type III row vector, the primed matrices are the prefactors on both sides of 
the equations. 


11.8 Minor Products of Type IV Vectors 


We have seen in Section 11.2 how a supermatrix may be expressed as à 
row vector whose elements are type III column vectors, or a column vector 
whose elements are type III row vectors. We have also seen how the major 
and minor products of type III vectors are formed. We shall now consider 
how the products of supermatrices may be expressed in terms of the 
products of the type III vector elements from type IV vectors. 


Notation for the minor product of type IV vectors 


First we shall consider the minor 
method corresponds to that used fo 
vectors; the minor product is form 
vector by a type IV row vector. 


Suppose we have given the supermatrix Y as in Eq. (11.8.1). 


product of two type IV vectors. The 
r simple vectors and lower type super- 
ed by premultiplying a type IV column 


BV Rhee ain ¥en 
bnYi = [ta RES. ces. „йш (11.8.1) 
$V Ка vus m m 
We know from Section 11.2 that Y can be expressed as a type ГУ column 
vector by Eq. (11.8.2). 
aY 
ыйы, = | e (11.8.2) 


aY Jien 
th sides of Eq. (11.8.2 
es us Eq. (11.8.3). 


Now let us premultiply bo 


) by the corresponding sides 
of Eq. (11.2.6). This givi 


Y 
tad EX wallon УЙ, = Хы Xa, XJ] s (11.8.3) 
wY Near 
Multiplying out the right side of Eq. (11.8.3) gives 


tod DX] loa Уел = (aX SY eg. Хь, ьа +... + tXo Yten’ 


(11.8.4) 
If we sum the products on the right side of Eq. (11.8.4) to get a single matrix 
Z, we have 


lX wa] [o Уел, = ui op (11.8.5) 
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The right side of Eq. (11.8.4) shows how the product of two supermatrices 
may be expressed as the sum of major products of type III vectors, just as 
we learned in Section 9.2 that the product of two simple matrices can be ex- 
pressed as the sum of major products of simple vectors. 

The left sides of Eqs. (11.8.3), (11.8.4), and (11.8.5) show that for the 
products of two supermatrices to exist, the postpartitioner of the prefactor 
must be the transpose of the prepartitioner of the postfactor. This rule is 
the same as for the minor product of type III vectors. The right side of 
Eq. (11.8.5) shows three important characteristics about the product of two 
supermatrices: (1) The prepartitioner of the product is the same as the pre- 
partitioner of the prefactor. (2) The postpartitioner of the product is the 
same as the postpartitioner of the postfactor. (3) The product tells us 
nothing about the common superorder of the factors. These rules, you will 
recall, are analogous to those for the product of simple matrices. 


Diagrammatic representation of minor product 
Figure 11.8.1 illustrates diagrammatically how the product of two super- 
matrices may be expressed as the minor product of type IV vectors. 


Г ] f e Cy Ca 


4 


by by bs | 
а, 1 
by mY tea! 
а» 
h; Yia’ 
bs at te 
аз 
| tad Xi, tad Xt, faci Xs | | 
б »aY lea! 
ta LX ia] lia lica 
C3 


e €» 


wY teat 
a СЕТЕ у 


аз 


Хь, 
Fre. 11.8.1 (Cont. оп next page) 


242 


а 


аз 


SIMPLE MATRIX COMPUTATIONS 


а» 


Хь, 

bi с bi C» m á ] 
bi 

H ewe ш аа К жиш 

bi Cy bi Co za b. m 


Fra. 11.8.1 (Cont. on next page) 


MULTIPLICATION OF SUPERMATRICES 


aj] Xo; os Y tea 


Fia. 11.8.1 (Cont. on next page) 


б» e b. C» b. сз ] 
a 
с HHH Ы i b : 
b Сі b» Co b» C3 
а аз а» 
bz [n b. C2 be Сз 
tar) X; tY tea’ 
Г bs a bs Со bs ©з 
а, а\ 
bs E HH bs HA НЕН, 
bs с | bs в bs Cs 
(2 
| i EH E 7 | 
= 
bs с\ ] bs C2 bs C3 
ETI «[TT] «rri 
bs bs bs 


244 


SIMPLE MATRIX COMPUTATIONS 


AX BY a Y tear 
Fig. 11.8.1 (Cont.) 
Numerical example of minor product 


A numerical example using the same partitioning vectors as Fig. 11.8.1 is 


given in Fig. 11.8.9. 
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Transpose of minor product 


The transpose of the minor 
by taking the transpose of bot 
the symbols in reverse order, 


product of type IV vectors may be indicated 
h sides of Eq. (11.8.4). On the left we arrange 
and take the transpose of each symbol. On the 
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right we take the transpose of each product term in the same way. This 
gives us Eq. (11.8.6). 
on Хы lon Y lea)’ = сеа X" lisa 
= каны, + ted Yor ыы, +--+. + ted Ybor Xia (11.8.6) 
Equation (11.8.6) can be expressed more compactly if we take the trans- 
poses of both sides of Eq. (11.8.5) to get 
teal Хел, = teaZ tan’ (11.8.7) 
We may illustrate the transpose of the minor product of type IV vectors by 
using the numerical example in Fig. 11.8.2 to get Fig. 11.8.3. 
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11.9 The Minor Product Moment of а Type IV Vector 


For type IV vectors, as for type III vectors, we must distinguish between 


the minor product moment of a row vector and a column vector. Cor- 


respondingly also, either a row or column type IV vector may, in simple 
form, be either a vertieal or horizontal matrix. 
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Minor product moment of type IV row vector 


First, let us consider the minor product moment of a type IV row vector. 
Here we must postmultiply the row vector byitstranspose. Taking the row 
form given by Eq. (11.2.6) and its transpose given by Eq. (11.3.10), we get 
for the minor product moment Eq. (11.9.1). 


tonlX wa’) [pa X" ten’ = (Хы Ky, жез ЖЫ 


Expanding the right side of Eq. (11.9.1) gives us Eq. (11.9.2). 


tonlX wal [oa = ta Xo X lor + Хь tX ln + 
sec (aXXo (11.9.2) 


ows how the minor product moment of a type IV row 


Equation (11.9.2) sh 
the major product moments of its 


vector may be expressed as the sum of 
type III vector elements. 
We may take the example for „Хы in Fig. 11.8.2 to illustrate Eq. 


(11.9.2). This we do in Fig. 11.9.1. 
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Fig. 11.9.1 (Cont. on next page) 
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It is clear from the final matrix in Fig. 11.9.1 that the minor product mo- 
Ment of a type IV vector is a symmetrically partitioned symmetric simple 
Matrix, 


Minor product moment of type IV column vector 


To get the minor product moment of a type IV column vector, we must 
Premultiply the column vector by its transpose. Taking the column form 
ae by Eq. (11.2.4), and its transpose given by Eq. (11.3.9), we get Eq. 

1.9.3). 
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7 11.9.3) 
ъа Хь = walXe. Xa o... XI] ( 


aX Jor 
Multiplying out the right side of Eq. (11.9.3) gives Eq. (11.9.4). 
| = Я , Жаа Ыр stes 
Фата) lX len = ы Хш Жы, + 3X7 Xi; 
4 + ХХ, (11.9.4) 
"The numerical procedure for getting the minor product moment of а у ре 
IV column vector as indicated in Eq. (11.9.4), is the same as for the minor 
product moment of the type IV row vector, as indicated in Eq. (11.9.2) and 
illustrated in Fig. 11.9.1. Notice, however, that in Eq. (11.9.2) we have the 
product of the supermatrix tX w,” postmultiplied by its transpose and ex- 
pressed as the sum of major product moments of its type ITI column vectors. 


In Eq. (11.9.4) we have the product of the supermatrix premultiplied by its 


transpose and expressed as the sum of major product moments of its type 
III row vectors. 


11.10 Major Products of Type IV Vectors 


Notation for major Product of type IV vectors 


To get the major product of type IV vectors we must premultiply a row 


vector by a column vector. From Eq. (11.8.1) we can express Y asa type IV 
row vector as in Eq. (11.10.1). 


ма) = ola Yo... y,] (11.10.1) 

We shall then have a major product of type IV vectors if we premultiply 
both sides of Eq. (11.10.1) by the corresponding sides of Eq. (11.2.4) as in 
Eq. (11.10.2). 

uX 

i 

Ге Хь У] = бА (Yu s Ys] 
aX Jy 


(11.10.2) 


Multiplying out the right side of Eq. (11.10.2), we get Eq. (11.10.3). 


Хь wal Yta] = 


aXpy Ya aX Bal’ bd Ye 


aX ta) ta Y e 
Хь t Y «y «Кы, tX. 


aX ta: pu Y c (11.10.3) 


„Хы ва „Хыны Хөл а, 
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The right side of Eq. (11.10.3) shows how the product of two super- 
matrices may be expressed as a matrix of minor products of type III vectors, 
Just as the product of two simple matrices can be expressed as a matrix of 
minor products of simple vectors (Section 9.2). 

ü The right side of Eq. (11.10.3) may be written more compactly as Eq. 

1.10.4). 


Go "WE. oan add 
aie жбер =з ape 

[a Xl ealY tea’) = є E d. Н v zà : (11.10.4) 
ut. ‘atop see. ales 


Numerical example of major product of type IV vectors 


We can illustrate Eqs. (11.10.3) and (11.10.4) by starting with the column 
form of X and the row form of Y of the supermatrices in Fig. 11.8.2. These 


We show in Fig. 11.10.1. 
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To illustrate how the minor products are formed from the type III row 
Vectors of X and the type III column vectors of Y to give the matric ele- 
Tents on the right of Eqs. (11.10.3) and (11.10.4), it will suffice to calculate 

€ first two elements of the first two rows. This we do in Fig. 11.10.2. 
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More simply we can write this as Eq. (11.10.6). 


4 1 
lea Y ңы, eX tar)’ = San HE PME (11.10. 
su uy eee ML 


If, then, we use the examples in Fig. 11.10.1, the matric element in the 
second row, first column, of Eq. (11.10.5) would be as given in Fig. 11.10.3. 
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As you would expect, the matric element we computed in Fig. 11.10.3 for 
the second row, fi 


rst column, is the transpose of the element; we computed 
for the first TOW, second column, in Fig. 11.10.2. 


11.11 The Major Product Moment of a Type IV Vector 


In view of what we know of type 3 vectors and the minor product mo- 
ment of a type IV vector, we may expect to have two kinds of major 
product moments of type IV vectors. 
product moment of a type IV row vect, 


Major product moment of type |V roy, vector 


First, let us consider the major product moment, of a type IV row vector. 
This implies that we premultiply the row Vector by its tranpose. Takin£ 
the row form given by Eq. (11.2.6) and its transpose given by Eq. (11.3.10); 
we get Eq. (11.11.1) for the major product moment, 
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С) 


Expanding the right side of Eq. (11.11.1) gives us Eq. (11.11.2). 
Хе Хь] = 


4 y 
aX lanl’ Хы ХлХ, ... aX lai tod Xo 

^ $ 4 
bX fag)’ Хы bX fag)’ ap] Xdy ... „Хш to Xo, (11.11.) 
„Хы tX bX lad (оь ... „Хш tap) Xo, 


We can illustrate а typical matric element on the right side of Eq. 
(11.11.2) by using the numerical example for the row form of X given in 
Fig. 11.3.10. The element in the first row, second column, of Eq. (11.11.2) 
would then be as indicated in Fig. 11.11.1. 
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You can readily show also that the element in the second row, first 
Column, is the transpose of the element computed in Fig. 11.11.1. 


Major product moment of type IV column vector 


To show the major product moment of a type IV column vector, we must 
Postmultiply the column vector by its transpose. Taking the column form 
8lven by Eq. (11.2.4) and its transpose given by Eq. (11.3.9), we get 
Eq. (11113). 
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Expanding the right side of Eq. (11.11.3) gives 
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To illustrate a typical matric element on the right side of Eq. (11.11.4), 


we may use the numerical example for the column form of X given in "s 
11.10.1. Then the element in the first row, second column, of Eq. (11.11. 
would be as shown in Fig. 11.11.3. 
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Note that in the practical applications of supermatrix multiplication in 
the social sciences, product moments of type IV vectors, either row or 
column, and major or minor, are more typical than the major or minor 
products of two distinct type IV vectors. Where the products of 
distinct type IV vectors are required, the factors may frequently be re- 
garded as submatrices of a larger supermatrix. It is also true that product 
moments of type III supervectors, either row or column, major or minor, 
are also more typical than products of distinct type III vectors. Where 
products of distinct type III vectors occur they are usually minor products 
and constitute the elements of the major product moment of a type IV 
vector. 


11.12 Тһе Product of Any Number of Supermatrices 


We have seen in Section 11.8 that for the product of two supermatrices to 
exist, the postpartitioner of the prefactor must be the transpose of the pre- 
partitioner of the postfactor. We have also learned that the prepartitioner 
of the product is the same as the prepartitioner of the prefactor, and the 
Postpartitioner of the product is the same as the postpartitioner of the 
Postfactor. Since these rules must hold for the product of any two super- 
matrices, they must hold also if we multiply the product of two matrices by 
another matrix. Therefore, we can indicate the subscripts for the product 


of three supermatrices as 


to toa’ tX tear ted Y teu? = ton) tay’ (11.12.1) 

We can readily generalize from Eq. (11.12.1) to the product of any 
number of supermatrices: (1) For the product of any number of super- 
matrices to exist, it is only necessary that for each pair of adjacent factors, 
the postscript of the prefactor be the transpose of the prescript of the post- 
factor. (2) The product of any number of supermatrices has the same 
Prescript as the left-hand factor arid the same postscript as the right-hand 
factor. | 

The transpose of the product given in Eq. (11.12.1) is given in Eq. 


(11.12.2). 


(Zi) = шл, = dY tea а Хы" oa Wian (11.12.2) 
We can easily generalize from Eq. (11.12.2) to the transpose of the 
Product of any number of supermatrices. Obviously, the rule is the same 
as for simple matrices. The transpose of the product of any number of 
Supermatrices is the product of their transposes in reverse order. 
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1. Notation for supermatrices: 
а. Prepartitioning. 
(1) The simple matrix Хо is partitioned into a type III column 
vector by the prepartitioning column vector prescript [a,], thus 
tenlPXo] = („Хо 
where 
а 
la] =| ^ 
a, 
(2) The number of matric elements in the type III column vector 


T x ; ic 
1 Хо is т, and the number of simple rows in the ith matri 
element is a;. 


(3) The sum of the elements in [a.] is 
l'a, = P 
b. Postpartitioning. 


(1) The simple matrix 


j , r 
»Xo is partitioned into a type III row vecto 
by the postpartiti 


oning row postscript [b,]', thus 
[PX gly = PX wy 
where 


[b.]’ = [bı (E ba] 


(2) The number of matric elements 


is s and the number of 
is b; 


in the type ITI row vector РХ! =. 
simple columns in the ith matric eleme 


(3) The sum of the elements in [b.]' is 


01 =Q 
c. The matrix PXQ is pre- 


: s 
and postpartitioned into r matric rows and 
matric columns by 


tanleX o]. = la] X [СА 
d. The ijth matric element of the 
2. The type IV supervector: 
a. The supermatrix fa X pay 
vector whose elements are 


? is «X 
Supermatrix „Хы, 15 «45 


n 
may be expressed as a type IV colum 
type III row vectors by 
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doe 


(ыйы = | 


aX ior 
b. The supermatrix (4j X p, may be expressed as а type IV row vector 
whose elements are type III column vectors by 
alama] = „Хы Xo, ... Xu] 
3. Transpose notation for the supermatrix: 
a. The transpose of the type III column vector is indicated by writing 
the transposes of the symbols in reverse order, thus 
„Хој = «Хы 
b. The transpose of the type III row vector is similarly indicated by 
[PX wal! = ХР 
c. The transpose of the type IV column vector is 
(tX = walX ton] 
d. The transpose of the type IV row vector is 
(Хъл), = [a Хш 


4. The minor product of type III vectors: 


a. For the minor product of type III vectors to exist, the postpart- 
tioner of the prefactor must be the transpose of the prepartitioner of 


the postfactor. 
b. The minor product of type III vectors is a sum of products of cor- 


responding matric elements from the two vectors, and hence a simple 
matrix, thus 
PX wy b) Ym = PX uYu +... + pXw y Yu = Pm 
с. The transpose of the minor product of type III vectors is the product 
of the transposes in reverse order, thus 
PX py Yar)’ = MY fo" o XP = мүр Xp Tee MY u Xp E “Zp 


5. The minor product moment of a type III vector 


a. The minor product moment of a natural order type III row vector is 
a sum of product moments of the matric elements of the vector in 


which the postfactors are primed, that is, 
PX pa waXe = pPXyo,Xp +... РХ Хь 


It is a simple symmetric matrix. 
b. The minor product moment of a natural order type III column 
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i e 
vector is a sum of product moments of the matric elements of thi 
vector in which the prefactor is primed, that is, 

MY four ым = uYnwYa +... + му Ум 
6. The major product of type III vectors 


+ M sas d 
a. The major product of type III vectors is a supermatrix partitione 
as the factors, thus 
XQ QV t? = ше, 


‚ В ts is the 
b. The element in the ith row and jth column of the шош be 
product of the ith matrie element from the prefactor postmultip 
by the jth matric element of the postfactor, that is, 


«ХооҮ,, = „2, 
с. The transpose of the major product of type III vectors is given by 
Хос] = УХ = „Ж 
7. The major product moment of type III vectors: 
a. The major product moment of a type III column vector is а sy™- 
metrically partitioned symmetric simple matrix given by 


ta) Xo QX fa,’ 
in which the ijth element is 


«XQ oX;,. (The postfactor is primed) 


A ; m- 
- The major product moment of a type III row vector is а SY! 
metrically partitioned Symmetric simple matrix given by 


wXp PX wr 
in which the ijth element is 


„Хь PX, (the prefactor is primed) 
8. Minor product of type IV vectors: 


a. The product of two su 


> іпог 
permatrices may be expressed as the mi 
product of type IV su 


pervectors, thus 

tan X ta] [oa Yea = „Хы, +... 4 (0X0, v, Y te = 0012010607 

b. For the minor product of type IV supervectors to exist, the РО 
partitioner of the prefactor 


r 
must be the transpose of the pre? 
tioner of the postfactor. 
c. The minor product of type IV 


tioned as the prefactor and p 
d. The minor product of type IV 


st- 
ti- 


x arti- 
vectors is a supermatrix P tob. 
ostpartitioned as the post wr 
Supervectors is the sum of ™ 
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products of type III vectors in which the ith term is the ith element 
of the prefactor postmultiplied by the ith element of the postfactor. 
е. The transpose of the minor product of type IV vectors is indicated 
by the transposes of the symbols in reverse order, thus 
Can LX toa] [a Y Tea!) = tealY tear] lea X" lon = tea tor! 


9. The minor product moment of a type IV vector: 


а. The minor product moment of a natural order type IV row vector is 
a sum of major product moments of its type III column vector ele- 


ments given by 
Хь [iba X" ед, = tap Xv Хд, +... naX Хд, 


b. The minor product moment of а natural order type IV column 
vector is a sum of major product moments of its type III row vector 


elements given by 
LX ta] [to Xo = aXX tar +... bX ar aX ta 
10. The major product of type IV vectors: 
à. The major product of type IV vectors is expressed as а matrix of 
minor products of type 3 vectors given by 
M» walYa Уш... Yel = Zee 


a,X 
[t Ж], bdl Yta] = | 


S^. lba’ 
b. The ijth element of a major product of type IV vectors is given by 
„Же, = aX tar b) Ve; 
c. The transpose of the major product of type IV vectors is written by 
taking the transposes of the symbols in reverse order, thus 
[sa Хы walY read}! = [ea Y lea’ walX tea] = teaZten 


11. The major product moment of a type IV vector: 
a. The major product moment of a natural order type IV row vector is 
a matrix of minor product moments of its type III column vector 


elements. It is indicated by 
[ea X Ite” tsi X wa] 


and its ijth element is 
dX lanl’ too; 


b. The major product of a natural order type IV column vector is a 
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pà 


æ 


© 


10. 


п. 


- Assuming all the followin: 
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matrix of minor product moments of its type III row vector ele- 
ments. It is indicated by 


[X tar АР. 
and its ijth element is 


; 
aX ta ы Ха, 


EXERCISES 


- Which of the following are type III row vectors and which type III column 


vectors? 


(a) ыт (b) Wia (е) ton We (d) ош 


= What is the simple height of each of the matrices in Ex. 1? 
- What is the simple width of each of the matrices in Ex. 1? 
- What is the superheight of each of the matrices in Ex. 1? 
. What is the superwidth of each of the matrices in Ex. 1? 


- Which of the following are incorrectly written? 


(а) ыы (ыы 


(е) ыы, (d) ton a 
(е) “Жы, (0) tag Woy 


(B) ton Win 


- Which of the following are square matrices in simple form? 


(а) ыы, (ушЫ, 


(е) uj Wi (d) ta) Wtan’ 
(е) мы, (ырш 


(в) wal fea 


. n zhi re 
g exist, which are type IV row vectors, and W hich а 
type IV column vectors? 


(а) lX (ЫХ 


(с) liXhay — (d) palXfaal 
() АХ) 0 [naX]. 


Remove the parentheses from each of the following: 


(а) бы, (Ы) (әу) (с) (Wi) — (d) (fay ML)’ 


Remove the parentheses from each of the follow: 


ing: 
(а) бышы)” (b) (ealX ea] 


t 
(©) aW — (d) Mend en) 
(a) Indicate the kind of product and the kind of vectors represented by 
Wien каМ 
(b) What is the ith term in th 


1 € sum indicated by this product? 
(c) Is the product a Simple or supermatrix? 
(d) What is its order? 


12. 


13. 


14. 


15. 


16. 


17. 


18. 
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Without the use of parentheses write the first term in the sum indicated by 
(oX tar tar) YP)’ 
If the zth term of a sum representing a product of supervectors of superorder r is 
оХ.; “Хо 


(a) What type of supervectors are involved? 

(b) What kind of product is it? 

(c) Is the natural order supervector, a row or column? 
(d) Indieate the notation for this vector product. 


The ith term of a sum representing a product of supervectors of superorder ¢ is 
PY; a YP 

(a) What type of supervectors are involved? 

(b) What kind of product is it? 


(c) Is the natural order supervector, a row or column? 
(d) Indicate the notation for this vector product. 


(a) Indicate the kind of product and kind of vectors represented by 


to] YP PX tn’ 
(b) What is the ijth element in the matrix indicated by this product? 


(c) Is the product a simple or supermatrix? 
(d) What do you know about its order? 


The ijth element in a product of supervectors of superorder r is 
a XQ «Ха; 

(а) What type of supervectors are involved? 

(b) What kind of product is it? 


(c) Is the natural order supervector a row or column? 
(d) Indicate the notation for this vector product. 


The ijth element in a product of supervectors of superorder t is 
«Ур Ye; 

(a) What type of supervectors are involved? 

(b) What kind of product is it? 


(c) Is the natural order supervector, a row or column? 
(d) Indicate the notation for this vector product. 


(a) What type of supervectors are involved in the product 
taU [tort Уел” 

(b) What kind of product is it? 

(c) What is the first term in the sum representing this product? 

(d) What type of vectors are involved in this term? 

(e) What kind of product is this term? 


(f) How many terms are there in the sum? 
(g) What is the order of the product? 
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19. Remove the parentheses from 


20. 


21. 


22. 


23. 


24. 


25. 


26. 


(eal X tort’) [sa Y tea?) 
(a) Write the product moment of a type IV vector if the ith term representing 
the product is 
Хь bX lar’ 


and there are s terms | 
(b) What kind of product moment is this? 
(c) Is the natural order of the vector a row or column? 


(a) Write the product moment of a type IV vector if the ith term representing 
the product is 
(64 X6; aX t 
and there are r terms. 
(b) What kind of product moment is this? 
(c) Is the natural order of the vector a row or column? 


The ijth element in a product of supervectors of superorder r X Ё is 


aX ty toa Yes 
(a) Indicate the notation for this product. 


(b) What type of supervectors is involved? 
(c) What kind of product is it? 


Remove the parentheses from 


(tea ы aX tn) 


The ith element of a product of supervectors of order s is 


ui 
Тр 


(а) What type of Supervectors are they? 
(b) What kind of product is it? 


Given the matrices 


Хеле ылы] = A 
and 


[oa Хь ьа] = B 


(a) What type of vectors is invol 
(b) What kinds of products? 


(c) What is the difference between 4 and B? 
Given the product 


ved in A and B? 


t ieai IYr a eZ y= tar) W tau] 


Fill in the brackets and put primes where needed. 


. (a) TypelV (b) Major product moment 


© lad X pay tal Y ted" ted Z tau 
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ANSWERS 


. (a) column (b) row (c) column (d) row 

- (a) 116] (b T (о (40 

3. (a) T — (b) la] () P (d) [а.]1 

. (а) s (b) 1 (c) r (d) 1 

. (a) ! (b) s (c) 1 (d) s 

3. (a), (b), (е), (f), and (g) are incorrect 

. (d), (e), (f), and (g) are square 

. (b), (d), and (e) are rows; (a), (c), and (f) are columns 

‚@ ы, (ы! (0) аә (D „Мл 

а) tlin] Ф) lea X’ par (©) lea Wen’ (d) walla] 


. (a) Minor product of type III vectors (b) Wei Mz (c) Simple 


A) QXL 


РХ, a “Хо 
‚ (a) Type III (b) Minor product moment (c) row (d) oXt ta Хо 
. (a) Type III (b) Minor product moment (c) column (d) РУ, tea YP 


- (a) Major product of type III vectors (b) „ҮР Хь; (c) Supermatrix 


(d) Superorder is s X 7, simple order is 14а.) X (031 


- (а) Type III (b) Major product moment (c) column vector 


(d) ta Xo eX fon’ 


- (a) Type III (b) Major product moment (c) row vector 


(d) tea Yt Yin 


‚ (à) TypeIV — (b) Minor (0 maXaaYtr @ Type II 


(е) Major (т (8) Хе’ 


* el Yt] Хь 

-&) о Хо [poX i Ф) Minor 
(а) walian] ы Хы, (0) Minor 

- (а) [ter Хы” palea] Ф) Type IV 


(c) Row 
(c) Column 


(c) Major 


= [raXliar pal Y tea‘) 
- (à) Type IV (b) Major product moment 


(c) In A the natural order is a 


row, in B a column 


= шї 


Chapter 12 


Simple Statistical Equations 


12.1 The Mean 


3 imple 
We shall now see how matrix notation enables us to аен een 
statistical concepts in convenient form. Perhaps опе of the simples 


‚ аа s 2 mbers. 
cepts in statistics is the mean, or arithmetic average of a set of nu 
Suppose we have a set of N measu 


them together and dividing by the 
seen, the mean of a nu. 


if we let the measures be the ele 


the relationship betw 
the matrix notation. 


N 
ух = X'in = 14x (12.1.1) 
1 


: ; ; the 
We notice that it does not matter which of the two vectors is used не to 
prefactor in getting the scalar or minor Product, since a scalar is equ 

its transpose. 


To indicate the number of measures, we have 


N = 11, gasa 
im ; 
Ordinarily, the mean is expressed as 
N 
Y а 
Мх = ух (213) 
N 
Substituting from the right sides of Eqs. (12.1.1) and (12.1.2), we have 
Mx = Xy 12.1.4) 
Il, 
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We could also express this as 
Mx = (Хх) (1515) = (1y1y)X"lu (12.1.5) 


You will notice that since the minor product moment of the unit vector 
to the —1 power is a scalar quantity, it is commutative with all other 
factors. Therefore, we can write it either before or after the minor product 
of the X vector by the unit vector. Since the primary purpose of matrix 
algebra is to simplify the analysis of data, we customarily do not indicate 
the number of measures by a minor product moment of the unit vector, 
We simply use the letter N to indicate the number of cases. Furthermore, 
since the unit vector must be conformable with the X vector, we do not 
need to use the subscript N to indicate the order of the unit vector. Ordi- 
narily, then, in matrix notation we would indicate the mean of a number 
of measures as 


^ 
Mx = zl (12.1.6) 
We could also write it as : 
Mx = n (12.1.7) 


122 The Variance 


If we let X be a vector of raw measures and Mx the mean, then the 
Standard formula for the variance, or the square of the standard deviation 
с of а set of measures is given as 

c A - ма (12.2.1) 


Now we have already seen in Section 8.5 that the sum of squares of a num- 
ber of measures is the minor product moment of the vector whose elements 
are those measures. Therefore, we can write 

УХ? = X'X (12.2.2) 
If now we substitute from the right sides of Eqs. (12.1.6) and (12.2.2) into 
the right side of Eq. (12.2.1), we have 


Ü XV 
а? = хх = (5) (12.2.3) 
But the second term on the right of Eq. (12.2.3) can be written as 
"nu IMEI 
(F) z Cz) пош 


And since the transpose of a scalar product of vectors is equal to the scalar 
Product, we have 
XT 
N 


+ 
-1% (12.2.5) 
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If we substitute the right side of Eq. (12.2.5) for the second factor on the 
right of Eq. (12.2.4), we have 


(29) p Gir) (12.2.6) 


Since matrix multiplication is associative, we can write the right side o 


Eq. (12.2.6) as 
XAY _ y(x (12.2.7) 
C) - x) 


Substituting from the right side of Eq. (12.2.7) into the second term of the 
right side of Eq. (12.2.3), we get 


"ep йк! UNE (12.2.8) 
е ос (Ge) 
Now the first term on the right of Eq. (12.2.8) can be written as 
XX (1l (12.2.9) 
ү Tx 


You notice that interposing the identity matrix between X' and X does Р 
alter the value of the product. Next let us substitute from Eq. (12.2.9) fo 
the first term on the right side of Eq. (12.2.8), and we get 


if I FL 2.10 
dx (5)х E x (5) (12.2.10) 


Since matrix multiplication is distributive, we know that the right side 
of Eq. (12.2.10) can be written as 


{ п [( 1r 2.11) 
(а) = (дв) х{(Шх-()х] c 
Again, since matrix multi 


plication is distributive, the factor in brackets оп 
the right side of Eq. (12. 


2.11) can be written as 


I 1 I 1 2.12) 
(х)х = (x 3 (2 E =x (12. 
If we substitute from the right side of Eq. (12.2.12) into the right side of 
Eq. (12.2.11), we have 


T eer nhi JI 2 2.13) 
(5) - x (m=z (у-м)х 022 
Substituting the right side of Eq. (12.213) into the right side of ЕФ 
(12.2.11) gives us 


(І Ц 
а х(% = mE (12.2.14) 
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Now from simple algebra we know that the term in brackets on the right of 
Eq. (12.2.14) can be written as 


I. UA a 11’ 
(5 = x) = x = x) (12.2.15) 


The factor 1/N is a scalar that is commutative with all factors; therefore, 
substituting Eq. (12.2.15) into Eq. (12.2.14) gives us finally 


БЕК Ht * 
с? = n> (1 N y (12.2.16) 


Equation (12.2.16) is simply Eq. (12.2.3) written in another form. For 
computational purposes one would use Eq. (12.2.3) rather than Eq. 
(12.2.16). We shall need Eq. (12.2.16), however, in the next section when 
we express the variance of a set of measures as a function of these values 


measured from their mean. 


12.3 The Variance of a Set of Measures from Their Mean 


The deviation score formula 


If now we let x be a deviation measure, the standard equation for x in terms 
of the raw measure and the mean of these measures is given In scalar 


notation as 

з= Х – Мх (12.3.1) 
Then the vector of deviation measures, if we let x be the entire vector 
rather than simply a single element of it, is 

x= X — 1Мх (12.3.2) 
where X now is a vector of raw measures rather than a single raw measure, 
We know that in terms of deviation measures the variance is given їп scalar 
notation by 


pem (12.3.3) 

ИШЕ; 
In matrix notation the variance in terms of deviation measures is given by 
сек (12.3.4) 

ig 


Proof of equality with raw score formula 


that the right sides of Eq. (12.3.4) and Eq. (12.2.16) 


Let us prove now 
р (12.1.7) into the second term on 


are equal. First, we substitute from Eq. 
the right of Eq. (12.3.2) to get "s 


z-2X-—1 N (12.3.5) 
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Equation (12.3.5) can be written as 


ll’ 36 
r-2lIX- vx (12.3.6) 
Applying the distributive law to Eq. (12.3.6), we get 
и (12.3.7) 
gc (1 у y 


We are now ready to substitute the right side of Eq. (12.3.7) into the T 
side of Eq. (12.3.4), but to do this we need the transpose of the right side 
of Eq. (12.3.7). We have, therefore, 


r= [(: - Х| (12.3.8) 


If on the right of Eq. (12.3.8), we take the product of the transposes 1n 
reverse order, we have 


ЛЕСИ 


But the transpose of a sum is equal to the sum of the transposes; therefore, 
we can write the second factor on the right of Eq. (12.3.9) as in 


„шу. PEN (12.3.10) 
(r=) -r-() 


But the transpose of the identit: 


3 ч , А е 
y matrix is the identity matrix and th 
major product moment of an 


y vector is symmetric. Therefore Eq. (12.3.10) 
may be written as 
шу lr (12.3.10) 
0307-0-39 


We could have obtained Eq. (12.3.11) directly by observing that the sum 
of any two symmetric matrices is a 


e 
symmetric matrix, and therefore th 
transpose is the original matrix. 

We can now rewrite Eq. (12.3.9) as 


[t xe] = x(1 i x) (12.3.12) 


Substituting Eq. (12.3.12) in Eq. (12.3.8) gives 


‚ TE 13) 
= у, 12.3. 
х = Х ( Xem ( 
Next we substitute Eq. (12.3.1) in Eq. (12.3.4) to get 


Й п’ il’ 
жї WV; _ 
ТЕ ( N X v) (12.3.4) 
ci AN CUN 
N 
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Let us now multiply out the two middle terms in the numerator in Eq. 
(12.3.14) as 


Li? 11“ Til’ 117 11^ /1r 
( alt 2 i; x * Gr) 


(12.3.15) 


Equation (12.3.15) may be greatly simplified if we remember that the 
product of any matrix multiplied by the identity matrix is the matrix 
itself. "Therefore Eq. (12.3.15) may be written as 


11’ n^ 1E firi ; 
(оа) 1-2 e RU) nuo 


Now the last term on the right of Eq. (12.3.16) may be written as 
WAA тал 
GG») == (12.3.17) 


But the middle product on the right of Eq. (12.3.17) is the minor product 
moment of the unit vector, which is a scalar equal to its order. In this case, 


the order is simply N. Therefore, we have 


11^ 11^ N 1 
IL WNI|sZiIdl-2-l! 12.3.18 
(rr) ^e" т! 55 


If we substitute the right side of Eq. (12.3.18) in Eq. (12.3.16), we have 
INC n^ -( - X) 12.3.19 
(00-8) -0-0) m 


Equation (12.3.19) can be written as 


(1 үз x) Ж (1 = x) (12.3.20) 


We have here a very interesting type of matrix that will often be useful. 
If we premultiply each side of Eq. (12.3.20) by the matrix on the right, 


we have 
1r na dle 9) 
( E 30 =, x) = (7 a (12.3.21) 


From Eqs. (12.3.20) and (12.3.21), we can also write 


( 2 x) s ( a 1) (12.3.22) 


In the same way, we can show that no matter to what positive integral 
Power we raise the matrix, it is still the same as the first power. Therefore, 
We have the general rule given by Eq. (12.3.23), where k is a positive 


integral power. 
LA 11^ 
(1 = x) - (! = =) (12.3.23) 
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A matrix is called idempotent if any positive integral power of it is the same 
as the matrix. : | 
Let us now substitute Eq. (12.3.19) into Eq. (12.3.14), and we get 


+ Pas 
e ax x y z zz (12.3.24) 


You see that Eq. (12.3.24) is the same as Eq. (12.2.16), which we derived 
by starting with the raw-score vector. 


The sum of deviation scores 


Let us note in passing that the sum of the deviation scores is 0. We know 
this is true by definition, but we shall prove it very simply by matrix 


algebra. We premultiply both sides of Eq. (12.3.5) by a row unit vector. 
This gives us 


Ve = "(х - ex) (12.3.25) 
From Eq. (12.3.25) we get 

Vom YX = 16:9 (12.3.20) 
From Eq. (12.3.26) we get 

Ve = xX – (27) "х (12.3.27) 


But we know that the factor in parentheses in the second term on the right 
of Eq. (12.3.27) may be given by 


n х N E (12.3.28) 
Substituting Eq. (12.3.28) in Eq. (12.3.27), we have 
Vem UX — 14x (12.3.29) 
Thus 
Их = 0 (12.3.30) 


We have therefore proved by matrix algebra that the sum of deviation 
Scores is equal to 0. 


12.4 The Covariance 


Matrix notation for covariance 


The covariance of two sets of measures, say x and y, is the mean of the 
sum of their products minus the product of their means, The scalar equ 
tion for the covariance is 
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If we let X be a vector of the X measure, and Y a vector of the Y measure, 
we have, in matrix notation, for the first term on the right of Eq. (12.4.1) 
ZXY ЖУ 

= 2 

ПЕ (12.4.2) 

The matrix equivalents of the factors in the second term on the right of 

Eq. (12.4.1) are given by Eqs. (12.4.3) and (12.4.4) respectively. 
УХ Al 


эх х (12.4.3) 
> ү 
zr = n (12.4.4) 


Substituting (12.4.2) to (12.4.4) inclusive in the right side of Eq. (12.4.1), 


we get 
ё: XE AITE 


= €— 
But the right side of Eq. (12.4.5) may be written as in 
TORT ny lyfr Шу 2 
сы = ixr- Р) = (1 зр) on 


Variance as a special case of covariance 
It is easy to show that the variance is a special case of covariance. If 
we substitute X for Y in Eq. (12.4.6), we get 


11 
mx 12.4.7) 
Crs = jx (1 xy ( 


But if you compare Eq. (12.4.7) with Eq. (12.2.16), you see that the two 
аге exaetly the same. 


12.5 The Pearson Product Moment Correlation Coefficient 


Scalar notation for the formulas 
There are а number of different so-called raw-score formulas for the 
Pearson product moment coefficient. of correlations. One of these is Eq. 


(12.5.1). 
i 2X; _ 2X; 2X 


ТОЕ г и NENNEN. SS аЕБЗ08 (12.5.1) 
zxi _ [>Х‹|' |223 _ EJI 
N N N N 


In this case, we have two sets of measures, X; and X; respectively. If we 
now multiply both the numerator and denominator by N, we get Eq. 
(12.5.2). 


(12.5.2) 
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If here again we multiply both numerator and denominator by N, we get 
Eq. (12.5.3). 
_ NIXA BII; 
VNZXi — [ZX } VEX? — [Хр 
Equations (12.5.1), (12.5.2), and (12.5.3) all give the same result. The my 
difference is in the method of computation. Equation (12.5.3) is sari 
by a good many computers because here the original measures are in who $ 
numbers, and we therefore do not need to deal with decimal numbers unti 
we begin to take the square roots of the denominator terms. On the other 
hand, if the number of cases is rather large, the first term in the numerator 


can become awkward to handle. It is not our purpose here to recommend 
any one of these three formulas above another. 


Tij 


(12.5.3) 


Matrix notation for the formulas 


Let us now write these three formulas in terms of matrix notation. We 
know that the sum of the cross products is the minor product of two vectors 
and that the sum of a set of measures is the minor product of its vector by 
the unit vector. "Therefore, we can write Eq. (12.5.1) in matrix notation aS 


Eq. (12.5.4). 
(12.5.4) 
Equation (12.5.2) in matrix notation becomes 
X'i 
ХАЖ 00 тр, 
у= X; N bg (12.5.5) 
For Eq. (12.5.3), we get 
"e NXX j- Xuüvx, (12.5.6) 
УУХХ = 


12.5.2), and (12.5.3) with the corresponding 
à Е » (12.5.5), and (12.5.6) leads us to conclude tha 
there is no partieular economy i 


; А : w 
coefficients in terms of the data matrix of га 
measures. 
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12.6 The Correlation Matrix from Raw Scores 


The raw covariance matrix 


Suppose we let X be a raw-score matrix of order N by r where N is the 
number of entities and n is the number of variables. In general, N is much 
larger than л, so the natural order of the matrix X is vertical. The first 
step in the computation of the matrix of intercorrelations among the n 
variables is the computation of the raw-score covariance matrix. This is 
simply the minor product moment of the matrix X divided by N. If we let 
G be this product moment matrix of raw scores over №, we have 


X'X 
G= N (12.6.1) 


Expanding the term on the right into a matrix whose elements are the 
minor products of columns of X, we have Eq. (12.6.2). 


ХАХА XXa ... XXa 
G=5 ХОХ. ХаХа о. ХЗХ (12.6.2) 
Жул Ка ee Xu 


The correction term matrix 


To get the correction terms for these raw-score product moments, we 
Shall need products of corresponding means. We therefore compute а 
vector of these means as in 


yy - iX = HK, vx. VK (12.63) 


To get the matrix of correction terms {ог the raw-score product momenti 
matrix, we need the major product moment of this vector of means. We 
indicate it as 


M = ҮмҮм (12.6.4) 


Then the covariance matrix for deviation scores is 
C-G—M (12.6.5) 
From Eqs. (12.6.3) and (12.6.4) we get Eq. (12.6.6). 
X^ 


1 X^ 
М = [Xa d'a e. ЖЫ] (12.6.6) 


Xl 
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The difference matrix 


Expanding the right side of Eq. (12.6.6) and substituting in Eq. (12.6.5) 
together with Eq. (12.6.2), we get Eq. (12.6.7). 


Жа хх» cee XN, 
Maka aX oon RON 
Pe hoe ee oe 
ў N 
Kal О Xa. ы. AX X. 
X4V EX, KANS x X3 1'X., 
Fo. Jagen. v. ur uu] oen 
N: 
Writing the right-hand side of Eq. (12.6.7) as a single matrix, we have Eq. 
(12.6.8). 


ZaXa ХЫХ. XUX. KX, хх, ХХ 
N BE ay ae - 


N? N? rae AMT N? 
ХХ 4 = X^ IX. XX. X41 VX XX, X41 И. 
XXX. Харх, Хох. XX, Хх X60 Xa 
» [ENS Ser xl. N 
(12.6.8) 


: me form as the ter, ; in the de- 
nominator terms of Eq. (12.5.4), ms under the radicals 


The diagonal matrix 


The next step is to make u а , 6.8), 
а Р р à matrix of th. . (12.6. 
which we indicate by Dc as in Eq. ( 1269) Pistes teas 
Dc = 5 
ХаХа ХХ, 
N ДЕ 0 47 0 
0 ХХ. Xavy 
ира 0 
0 0 хх  Xt1 Xo 
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But Eq. (12.6.9) is simply a diagonal matrix whose elements are the vari- 
ances of the variables in the data matrix X. We can therefore write it more 
simply as Eq. (12.6.10). 


сі Oy asa то 
Zee ын (12.6.10) 
0 би in. oe 


If now we let the numerator terms in Eq. (12.5.4) be represented by С 
Eq. (12.5.4) can be rewritten as 


pga (12.6.11) 
Oi; 
Also we can write Eq. (12.6.8) more compactly as Eq. (12.6.12). 
ec Cim ea Cin 
=| 9 бь (12.6.12) 
Cin да n LA 


Next let us take the square root of both sides of Eq. (12.6.10). This gives 
Eq. (12.6.13). 


a 0 n 0 

D = 0 о... 0 (12.6.13) 
0 UE к 

Then we take the inverse of both sides of Eq. (12.6.13) to get Eq. (12.6.14). 
ra eed 
сі 
1 

Dg? = ПЕТ " (12.6.14) 
; H 
0 O0 ze 


Multiplication by a diagonal matrix 
Next we premultiply Eq. (12.6.12) by the matrix in Eq. (12.6.14) to 
Веб Eq. (12.6.15). 


1 0 0 ep cO. x Um 
91 
1 C 2 C. 
7; sei ac * | (12.6.15) 


Реб = 
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Multiplying out the right side of Eq. (12.6.15), we get Eq. (12.6.16). 


C» Cin 
eS ш 2 
ei ei 
Ci Cos 
— a ... — 
Dg^C = | o 0» (12.6.16) 
Cin Con 
<ir Mida on 
On On 


Finally, we postmultiply both sides of Eq. (12.6.16) by the matrix in Eq. 
(12.6.14) as in Eq. (12.6.17). 


|| ee ee 
с сі с 
Съ Con 1 
[Dg^C]Dg/? = | о» Ree ea: [3 o E 0 
1 
Cin Con On 0 О 
Wh a on 
(12.6.17) 
Multiplying out the right side of Eq. (12.6.17) we get Eq. (12.6.18). 
‚| &@ . & 
9172 ^" gig, 
Sa ы 
[Lig?C]| Dg"? = | 010 "T area, (12.6.18) 
E Cu 1 
S104. Goo, 777 


But you see that the nondiagonal terms of Eq. (12.6.18) are the correlation 


coefficients as given by Eq. (12.6.11). Therefore the right side of Eq. 
(12.6.18) can be written as in Eq. (12.6. 


19). 
1 d эз» Жї 
[Dc ^C] Davus PNE Ki roe fas = (12.6.19) 
һә» ы D. i 1 


Alternate matrix formulas 


It should be noted that the entire development from Eq. (12.6.1) through 
Eq. (12.6.19) is based on the scalar formula, Eq. (12.6.1), for the correlation 
coefficient. You can easily see that if the Procedure had been based on the 
formula of Eq. (12.6.2), the steps would have been as follows: first, the 
covariance matrix C would have been replaced by N times this covariance 
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matrix as in Eq. (12.6.20). Then the diagonal term of the matrix NC would 
have been Dyc. 


NC = NG— Vyl'X (12.6.20) 
Corresponding to Eq. (12.6.19), we would have had 
IDRE NCIDRE = рор! (12.6.21) 


То prove that Eq. (12.6.21) is an identity, we first observe that the diagonal 
matrix of NC is N times the diagonal matrix of C. This is indicated in 


Dxc = NDc (12.6.22) 
If we substitute the right side of Eq. (12.6.22) into the left side of Eq. 
(12.6.21), we have 
(ND) NC] [ND] = De'"CDc'* (12.6.23) 
But notice that the terms in brackets to the — 1/2 power in Eq. (12.6.23) 
may be written as 
[үс]! = N-? Dei? (12.6.24) 
Substituting the right side of Eq. (12.6.24) into the left side of Eq. (12.6.23), 
we have 
[QN 2 Dz)NC] [N-? Dc?] = De! CDo? (12.6.25) 
Now since N is a scalar quantity, it is commutative with the matrices on the 
left of Eq. (12.6.25), and we can rewrite Eq. (12.6.25) as 
Ne NEN DAPCDg! = Dg CDgi (12.6.26) 
Obviously, the N's cancel on the left of Eq. (12.6.26), so the equation is an 


identity, Ў 
If a had used Eq. (12.5.6) as the basis for our computation, then the 
numerator term would have been N? times the covariance, and we would 


have had Eq. (12.6.27) instead of Eq. (12.6.19). 
[Disc] [N?C] [Donc]? = т (12.6.27) 


It is easy to prove that Eq. (12.6.28) is an identity. 
[Disc]? = N^! Dc (12.6.28) 

Therefore, we can rewrite Eq. (12.6.27) as 
N-Dg'2N?2CN De"? =r (12.6.29) 


From this we get 
Dg^CDg = т (12.6.80) 
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12.7 The Covariance Matrix from Deviation Measures 


Equation (12.3.7) shows a vector of deviation measures as a function of 
the vector of raw measures. Suppose now we wish to express an entire 
matrix of measures as deviation measures. We can write the same equation 


as Eq. (12.3.7), except that now X and z are of order N by n. We let this 
deviation score matrix be represented by 


pies (- Y) (12.7.1) 


Now the deviation covariance matrix would then be the minor product mo- 
ment of z divided by the number of cases. 


з= -c (12.7.2) 
But Eq. (12.7.2) can be written as 
хх = NC (12.7.3) 
one the right side of Eq. (12.7. 1) into the left side of Eq. (12.7.3) 
х1 - y) = Хх = NC (1274) 
N 


But because of Eq. (12.3.19), we can rewrite Eq. (12.7.4) as 


ll’ 
Tr - 5 
x(r y)x = NC (12.7.5) 


SUMMARY 


1. The mean of a vector of measures is expressed as 


IX x" 
Macc. 
" N N 


2. The variance of a vector of raw Scores, X, may be expressed as 


ot = хх ( 7 vx 
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3. The variance of a set of measures from their mean: 
а. A matrix of deviation measures x may be expressed in terms of the 


rawscore data matrix т by 


= (1 = A) 


n 
b. The matrix 7 — E is idempotent, that is, 


(1-3) - 6-3) 


for positive integral values of К. 
€. "Therefore, 


4. The scalar covariance C: . 
a. In matrix notation the scalar covariance Cz, of the vectors X and Y 


may be expressed as 
u fe шу 
= sax = Aa) - $Y (1 N X 
b. The variance is a special case of the covariance when X = Y. 


5. The Pearson product moment correlation coefficient may be expressed in 
matrix notation as a function of the raw-score vectors X, X; as follows: 


xix, _ X1 VX; 
—À m 
Mh Re, xürx |XX; _ XA vx, 
JE N NNN NN 

xu vx; 


XixX,— р: 


—— ———————————— 
ЕТТ, TAs; 
Jea- EE Ix - ат; 
"= = ANXOG — Ха VX: WNXGXs — ХХ; 


б. The correlation matrix from raw scores: 
a. The raw-score covariance matrix of the data matrix X is given by 
XX 


N 


b. т; = 


b. The vector of means is given by 
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<] 


to 


p 


ES 


ex 


d 


- (a) If V is a vector of measures of order N, 


- Which of the follow: 


d We = ыры 


ГА 
‚ (a) HNC = х(1 ке, 
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c. The covariance matrix is given by 
C=G—-VuVy 
d. The correlation matrix is given by 
r = De'?CDe'2 
where Dc is the diagonal of C. 


Р А Т 
. The covariance matrix may be expressed as a function of a matrix г о 


deviation measures by 


das 
dius 
EXERCISES 


"Io М ee 
what is че” 


(b) Write the same quantity another way. 


‚ (a) What ic у, ( = F) y? 


(b) Remove the brackets from the expression. 
Which of the following is the best computational procedure? 
(a) (V'1)Q'V) (b) V'ta15v] (е) [V'11]V 


ing is the best computational procedure? 
è 1r ү fir 
(a) v[(r- кү| (b) vv - vy 
VANS ^ y' 
(с) WV – (52): к ay рр xw ЧУ 


What аге 


4 ipd H’ 
(a) i : 5) (b) (1-2) 


if all unit vectors are equal? 


(a) If V; and V; are vectors of order N, what iss (1 - +) y? 
(b) If V; = У, what is it? 
N +; in terms of vectors of X? 


N үг what is NCi; in terms of the vectors of X? 
(b) What is C,; called? 
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. ЕС = — (1 = Tyr and r = DegCDov? 


(a) Write r;; in terms of the vectors of X without the use of brackets. 


(b) What is the diagonal D, of 7? 


ANSWERS 


- (a) The mean of the measures 


Y^" 
Wy 
- (a) The variance of V 
(b) avy twit ү 


- (a) 
+ (е) 
- (а) A row null vector 
(b) A column null vector 
9 XX; 
N 


(а) x ( - Ty 


(b) The variance of Xx 
кух Xurx, 
NUT NC ОМ 


XX, KLUX, [ХХ XA UX; 
бту x Na N? 


+ (a) м; = 


(b) 7 
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Chapter 13 


Computíng the Correlatíon Matríx 


13.1 The Role of the Correlatio 


n Matrix in the Analysis of Experimental 
Data 


In the analysis of data involving large numbers of variables or Баа 
the correlation matrix plays a fundamental role. As we have seen = a 
most of the problems in the behavioral sciences do involve large number E 
attributes, or to put it another way, data matrices with many oe 
Since the correlation matrix is so important in all problems pene nd 
analysis and prediction in the Social sciences, it is well for you to unders eds 
how it may be derived from a data matrix with least labor and greates 


cannot be clearly indi 
efficiently and economically designed. 
Correlation progr: 


ce it 
ams now available for high-speed computers make 
possible to calculat 


Е 5 with 
e large-order correlation matrices at great speed v 


can be used to compute the correlation 
most efficient manner. It wil] also sho 


the associative and distributive laws of 
the successive Steps in the computati 


+ in the 
: dxint 
matrix from the data matr ito 
W how, in terms of matrix no 


hec 
matrix algebra may be used to с 
on. 


13.2 The Data Matrix 


The row and column sums 


The basic data for the computatio: 
ments of the data matrix, or the mat 
of a given number of entities on eac! 
are given the data matrix X ‚ We со 


H ele- 
n of the correlation matrix are T 
rix of elements giving the value ton we 
h of a given number of attributes: ddin£ 
nstruct а supermatrix from it by 9 
286 
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one row and one column. The new row is simply a vector of column sums of 
X, or X premultiplied by the row unit vector. The new column is simply a 
vector of row sums of X, or the matrix X postmultiplied by a column unit 
vector. The element in the new row and column position is the sum of the 
elements in the new column, or the new column vector premultiplied by a 
row unit vector. The order in which each element of the supermatrix is 
obtained is shown in Fig. 13.2.1, steps 1 to 4. 


(X) 
VX] 1'(X1) 
2 4 5 
Fig. 13.2.1 


Checking the row and column sums 


As a final check operation on row and column sums, we postmultiply the 
column sum vector by the unit vector and write it outside the matrix beside 
the element in the last row and last column, as step 5 in Fig. 13.2.1. As you 
See, this is the same value as the fourth entry in the matrix except for the or- 
der of multiplication. The two must be equal since matrix multiplication is 
associative, If the two are not equal, we may assume that а computational 
error has been made in one of the steps. The error may be in steps 2, 3, 4, or 
S or in several of these. In order to find out which step contains the error, it 
is well to recompute first the step requiring the smallest number of opera- 
tions, Ordinarily, this would be step 5, which requires summing the ele- 
ments in a vector whose order is equal to the number of attributes, whereas 
Step 4. requires summing a vector whose order is equal to the number of 
entities, generally much larger than the number of attributes. 

If recomputing step 5 gives the same results as before, step 4 is repeated. 

again, the results are the same as before, we must assume that the TOW 
Vector of column sums in step 2 or the column vector of row sums in step 3 
18 incorrect. If the computations are made on а calculating machine, the 
number of keyboard entries for both 2 and 3 will be the same, since each 
element of X is punched into the keyboard once for both steps 2 and 3. In 
Step 2, however, the keyboard must be cleared only m times; in step 3, it 
Must be cleared n times. Furthermore, in step 2, only m values are copied 
from the dials to the paper, while in step 3, n values are copied; therefore, 
the actua] number of operations in step 2 is less than in step 3. We recom- 
mend, therefore, that step 2 be recomputed first. Finally, then, if necessary, 


8tep 3 would be recomputed. 
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13.3 The Minor Product Moment of the Data Matrix 


The formula 

The formula we shall consider first for calculating the elements of the cor- 
relation matrix is Eq. (12.5.6) in the previous chapter, that is, 
И NXX,- XX, 
—2XNXX.— XA TX; ANX:X; - XA VX; 
The general matrix form for the scalar quantity given by Eq. (13.3.1) must 


be indicated in several steps. Welet Eq. ( 13.3.2) be a matrix of numerator 
terms of the type given in Eq. (13.3.1). 


Tij 


(13.3.1) 


H = NX'X — (X’1)(1'X) (13.3.2) 
Then the matrix of correlation coefficients will be given by 


т = ртенрт^ (13.3.8) 


15. 18.2.1. Wethen premultiply this supermatrix by its 
transpose, or to put i 
this supervector, T 


X' 
— |[X | xy e 
Tx" 


The computational steps 


The computations are made in the following steps indicated by the bol E 
face numbers: Step 1 The mi 


Step 2 The sum vector is premultiplied by 


X' to give the column vector 0 
18 the second element in the 


first row of the supermatrix. Step 3 The thir 


tin 
"Therefore the first elemen 
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the second row need not be calculated, since it is the transpose of the second 
element in the first row. 

The fourth step is to multiply the supermatrix on the right side of Eq. 
(13.3.4) by a type I supervector whose first element is a unit vector, and 
Whose second element is a 0 scalar, as shown in Eq. (13.3.5). 


1 2 4 
xx | x' xi) J|? X (xl 
d Riego 6 (13.3.5) 
(X^)X| (1/X’)(X1) | LO 5 
3 


This, of course, amounts to summing the simple rows of the first column of 
the supermatrix product moment. Since, however, we have not entered the 
infradiagonal elements, our actual computation in Eq. (13.3.4) must con- 
sist of adding down a column and then across the corresponding row to get 
the element corresponding to the sum of elements in a given row. 

Figure 13.3.3 shows diagrammatically how these additions are performed. 
The elements not entered are indicated by the crossed-out squares. 


1 2 4 


B 
X 


x 


Fig. 13.3.3 


Checking the computations 

To check our computations, we compare the vectors obtained in steps 2 
and 4. If these two are equal, we can assume, except for possible compen- 
Sating errors, that all operations have been correctly performed. If one or 
Several corresponding elements in steps 2 and 4 differ, we cannot tell 
offhand whether the error is in 2, in 4, or in the rows of 1 from which 4 was 
Obtained. If 3 and 5 are equal, then we may assume that 2 is correct. If 
hey are not equal, however, we first recompute 5. If it has been correctly 
Computed, then we recalculate 3. We also calculate 6, which is the sum of 
he elements in the vector given by 4. This obviously should also be equal 
to3 and5. If itis equal to 3, we can assume that 4 is correct, and if 4 is cor- 
Tect, we can also assume that 1 is correct. However, if we have verified that 
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3 is correct and that 6 has been correctly computed from 4, then if 3 and 6 
do not agree, we know that 4 is incorrect. If 5 does not agree with 3, we 
know also that 2 is incorrect. 

Our next step, then, would be to check each element in 4 by resuming the 
corresponding row in 1 that does not agree with the corresponding element 
in2. In this way, we may narrow down the errors by correcting errors that 
might have occurred in computing 4. 

Still assuming that 5 and 3 do not agree, we may next recalculate the ele- 
ments in 2 that do not agree with the elements in 4. In this way, we correct 
the elements in 2 that were incorrectly calculated. With these corrections 
made and 5 recomputed, 3 and 5 should now check. We may assume then 
that all remaining errors are in the product moment matrix or 1. 

If now there are only two remaining errors in 4, we check the element in 1 
corresponding to the row and column indicated by the two errors in 4. For 
example, if the second and fifth elements of 4 are incorrect, then we might 
assume that the error would be in the second row and fifth column of 1. 
This we would recalculate, However, if there is only one error in 4, this i$ 
almost certain to be in the corresponding diagonal element since any non- 
diagonal error would involve two errors in 4. Conversely, of course, We 
could have two errors in 4 simply by having errors in the corresponding 


diagonal elements. In the case of two errors, however, it is better to check 
the corresponding nondiag 


resp onal elements before checking the two corre- 
sponding diagonal elements. 


In case of more than two errors in 4, it is well to use the scheme illus- 
trated in Fig. 13.3.4, 


Fic. 13.3.4 


Draw a grid on graph pa 


É nx 
Indieate on a co! Per corresponding to the product-moment mat 
n H 


à ed. 
umn to the right the position in which the errors are loc 
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Draw a dotted line through each row corresponding to the position of an 
error and then draw dotted lines through the corresponding columns. Each 
row corresponding to an error will then have in it as many intersections of 
dotted lines as there are errors, including the diagonal. 

Now the first step, beginning with the first error, is to check the entry 
corresponding to the first nondiagonal intersection in the corresponding 
row. If an error is found in this position, we correct it and recompute the 
two corresponding elements for step 4. For example, if the second element 
in the first row of Fig. 13.3.4 has been found to be incorrect, and if the cor- 
rect value substituted in the matrix 1 of Eq. (13.3.5) makes the first element 
of 4 agree with the first element of 2, then we may assume that only one 
error was made in the first row. But this was also an error in the second row, 
since it was in the second column of the first row. We therefore recompute 
the second element in operation 4, and see whether this now checks with the 
second element in 2. If so, we may Mow assume that all circled positions in 
the first and second rows and columns are correct and exclude from further 
consideration the first and second rows and columns of the grid in Fig. 
13.3.4. We then proceed to the next uncorrected error in the same way. 

Let us assume, however, that the correction of this first nondiagonal ele- 
ment does not result in agreement between the first elements in steps 2 and 


4. We then go on to the next intersection, which in our example is the one 
in the fourth position, and recompute it. Then we recompute the first entry 
ve continue to check all possible errors in the first 


in step 4. In this way, У s ‹ 
TOW and make the necessary corrections below the diagonal in each case of 


error. If all nondiagonal intersections in the first row turn out to be correct, 
then we check the diagonal element. The advantage of checking the non- 
diagonal elements first is that errors in these positions, if corrected, are 
likely also to correct other diserepancies in the step 4 vector. Once all 


errors in the first error row have been corrected, it is quite possible that one 


Ог more of the subsequent error rows will check. For all errors eliminated in 
ted from the grid in 


the first row, corresponding rows and columns are dele i 
Fig. 13.3.4, and the same process is repeated for subsequent rows that still 
fail to check. 

The total number of discrepancies in steps 


greater than the number of errors in the row of 4 t the n 
number of errors. As skill in the operation of computing machines increases, 


the number of errors will decrease, and the operations suggested in connec- 
tion with Fig. 13.3.4 for finding the errors will probably reveal but a single 
error in any one row or column. For à reasonably skillful operator, the 
diagonal elements are less likely to be incorrect than the nondiagonal, be- 
cause the former is a sum of squares and the latter a sum of cross products. 
It is then to be expected that in general the correction of one error will 


Correct two discrepancies in steps 2 and 4. 


2 and 4 will be equal to or 
f 1 that has the maximum 
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13.4 The Correction Matrix 


The computations 


machines. We assume that the vector of column sums is available € 
Fig. 13.2.1. Using the last row of the matrix in Fig. 13.2.1, we calculate th 2 
major product moment of this supervector as in Eq. (13.4.1). As in Warn 
the product moment matrix of X, we do not write down entries below the 
diagonal, but use the same scheme as shown in Fig. 13.3.3. 


1 if 2 
T [X | VXI] = (X'1)'X) (X'1)(1X1) (13.4.1) 
Dal ах) ахъуах1) 

3 


The computational checks 


To get our check column from E 


š irst 
Ч. (13.4.1), we sum rows in the firs 
column of the right-hand side of Eq 


- (13.4.1), as in Eq. (13.4.2). 


1 2 

(X'1)(1'X) (X’1)(1'X"1) 1 
(1'X"1)(1/X) (UX'1)(1'X1) |01 = 
3 


4 


099009) О) хур) 
6 


(13.4.2) 
XIU) 
5 
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recalculation of 5 should check with 3. We next proceed to recalculate those 
elements of 4 that differ from the corresponding elements in 2. If any of 
these have been incorrectly calculated from 1, we correct them. 

We are now safe in assuming that the remaining discrepancies between 2 
апа 4 are due to errors in 1. The procedure for locating errors in 1 is the 
same as the procedure for locating errors in the minor product moment 
matrix of X. We identify the rows in which the errors are located and use а 
grid with dotted lines as in Fig. 13.3.4 to locate the errors in step 1 of Eq. 


(13.4.2). 


13.5 The Difference Matrix 


The computations 


First we multiply the matrix given on the right of Eq. (13.3.4) by the 
Scalar quantity N. This operation is shown in Eq. (13.5.1). 


Xx | X'(X1) N(X'X) | N[X'(X1)] 
(13.5.1) 


N 


axx| axon) | Luta'xox MUXX 


To check these computations we postmultiply the right side of Eq. (13.5.1) 
Y а supervector whose first element is the unit vector and whose last ele- 
ment is a 0 scalar, as in Eq. (13.5.2). 


1 2 4 
N(X'X) | NI[X'(X1) [| | [N(X'X)]1 | 
NEXX | viaxan) | LO rN 
3 


МОИ) (3.52) 


Next we multiply the vector obtained in step 4 by the unit vector to get the 
Scalar in step 6. If step 6 is equal to step 3, we may assume that the compu- 
tations are correct. (As in the computations of steps 3 and 4, we do not have 
entries below the diagonal.) If 5 and 6 are the same, we may assume 2 and 4 
are the same. The simplest way to check 1, assuming that 2 and 4 do not 
Agree, is as follows: first recalculate each element in 2 that disagrees with its 
Corresponding element in 4. Then recalculate each element in 4 that differs 

Tom its corresponding element in 2. Next use the grid system represented 


Y Fig. 13.3.4 to find errors in step 1. 
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Next we subtract from the matrix on the right of Eq. (13.5.1) the matrix 
on the right of Eq. (13.4.1). This is illustrated by Eq. (13:5.3). 


N(X'X) | NIX'(X1)] | E [ DUX) | ax'Daxa) | = 
Кс N(1'X')(X1) a'x'Da'x)| (1X1) (1X1) 


N(X'X) – (ХИХ) N[X'(X1)) — (X"1)(1’X"1) | (13.5.3) 
mis = a'x'na'xy|xa'x?on — (XY) 


Let us now define new symbols by Eqs. (13.5.4) and (13.5.5). 
G-NX'X (13.5.4) 
L = (Х'1)(1'Х) (13.5.5) 
Then Eq. (13.5.3) can be written as Eq. (13.5.6). 


ват POP ИШЕ ae (13.5.6) 
VG voi UL YD [rG—gaL ver -rr 


Again, for the matrices in Eq. ( 
the diagonal. 


13.5.6) we do not write in the entries below 


The computational checks 


4 
1 2 25 м 
G-L G1 — Іл г = oria: um VG = 1)1) (13.5.7) 
I'G — VL 0 


101 — rz QG — VL) $ 
3 5 
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that the infradiagonal elements have been properly copied from those above 
the diagonal. Having verified the accuracy of the copying, and made the 
necessary corrections in both 1 and 4, we proceed to locate the remaining 
errors in 1 by testing those elements isolated by means of the grid technique 
in Fig. 13.3.4. 

It should be noted that the checks used to verify the computations for the 
products of matrices are based on the associative law, whereas those used to 
verify the computations for the difference between matrices are based on the 
distributive law. 


13.6 The Simultaneous Computations for the Difference Matrix 


The computations 


As stated in Section 13.4, it is not necessary to go through the three 
Separate operations indicated by Eqs. (13.4.1), (13.5.1), and (13.5.3) to 
£et the H matrix. We could immediately set up the product of two super- 
matrices, as in Eq. (13.6.1). 


X'X| (X'X)1 
INI | — x^] = 
VX | TX 


[N(X'X) = (X'DQX). NXX) — X'07X1) (13.6.1) 


Considering first the left-hand factor of the product on the left of Eq. 
(13.6.1), we sec that the first term is simply a scalar matrix NI. The 
Second term is a column vector of column sums obtained from step 2 in Fig. 
13.2.1. Considering next the right-hand factor on the left of Eq. (13.6.1), 
the first row of this supermatrix is simply the first row of the right-hand 
Side of Eq. (13.3.4), which we assumed to have been checked by Eq. (13.3.5) 

he second row of this matrix is simply the second row of the matrix in 
Fig. 13.2.1, which we presumed also to have been checked. 

If now we sum rows of the left element in the type III supervector on the 


right of Eq. (13.6.1), we get Eq. (13.6.2). 
IN (x'x) oo | N(X’X)1 P о H Et 
3 
[N(X’X) - (Х'1)(1'Х)1 (13.6.2) 


But note that here the right side of the equation is the same as the second 
element of the first factor on the left, except for order of multiplication. 

ecause of the distributive law, the third operation in 3, if it agrees with the 
Second operation, checks the first set of operations. 
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13.7 The Case When N Is Very Large 


The formula 


1 
If the number of entities happens to be very large, then ads d 
(Eq. 13.3.1) would not be advisable as а basis for computing the corre 
matrix. In this case, the preferred formula. would be Eq. (13.7.1). 


1х; 
AX. — (e 


fü — —————— SNS NEU 
d UX; XIX 
vxo - HP fg, ‚айти 


N 


(13.7.1) 


be 
Then the matrix form for the numerator terms of Eq. (13.7.1) would 
ia (13.7.2) 
my — , ia 
Kx (X X s) 
The matrix of correlation coefficients would be 


r= Dg? K Dz? (13.7.3) 
= : by 
Obviously, the relationship between Eqs. (13.3.2) and (13.7.2) is given 


74) 
H-NK (18 


The vector of means 


If Eqs. (13.7.1) and (13.7.2) are used a: 
tion matrix, then we still use E 
product moment of X, but we 
tion matrix and its check requ 
indicated by the Second fact; 
Eq. (13.7.2). This vector w 


в à basis for computing the со 
qs. (13.3.4) and (13.4.5) to get the мү 
cannot use Eq. (13.4.2), because the corr! a 
ire that we first compute the vector of E of 
Ог in the second term on the right side 


wx tex ay. (exer. (13.7.5) 
[alex iru - eS] 
The very simple check for this division is given by 
E | UX1]n1] frx (13.7.6) 
WF оъ 


tal 
This simply means that the sum of the means must be equal to the ae 
sum of all of the elements in X divided by N. In other words, the Г 


i ‚ first 
hand side of the equation must be equal to the second element in the 
factor of the left-hand side. 
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The correction matrix 


The correction matrix and its check is now given by Eq. (13.7.7). 


ars?) | eeu? 


EMEN 

, N N " 1 

rxi РА 
ax (5) ax »( x ) 


(13.7.7) 


As in the preceding product moment matrices, only the entries above the 
diagonal are written into the right side of the equation. The check for this 
equation is Eq. (13.7.8), and the interpretations are the same as the checks 


for Eq. (13.4.2). 


1 2 Е 
526.3 oro ( E [cv 
qx (Т^) (^X 1) A) { [exo] 

5 


vio) |n (13.7.8) 
6 


The difference matrix 

To get the difference matrix, we require no operations comparable to 
those indicated in Eqs. (13.5.1) and (13.5.2), because the correction matrix 
may be subtracted directly from the minor product moment of X without 
multiplication by a scalar. We proceed, then, to find the difference equation 


directly, as in Eq. (13.7.9). 


Xx | xag aea) | ae (s^) 
Q'X^x | а'х'ух1) 3 ax 0(4F) cx (47) 
dicla ae» ( e) ХХ 1) - oe» ( 


; я j VXI 
WXX - ax 263 q'x?) 1) - ax (F) 
(13.7.9) 
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The checks are then carried out as Shown in Eq. (13.5.7) for Eqs. (13.5.3) 
and (13.5.6). 


Direct computation of the K matrix 


If we wish to calculate the K matrix given by Eq. (13.7.2) directly, with- 


out first computing the correction matrix, we set up an equation comparable 
to Eq. (13.6.1) as 


X'X | (X'X) 


Z|- xa] 


(13.7.10) 
[xx = X^( 5 | ox = x (A) =([K|K1 


13.8 Calculating the Red d Mi ix Directly 
from the Date Mae Minor Product Moment Matrix 


IE we wish to calculate the matrix K given by Eq. (13.7.2), rather than the 


trix H gi : ind 
Ea. een by Eq. (13.3.2), we can set up the product matrix given bY 
MN ou X Xx - (5) 
EXE 


^v „ү! a 9:6 , 
VX"! yxy = rex - onay( 1) 


= n element s the type IIT Yector on the right of Eq. (13.8.1) is i 
reduced minor product moment iven b k is ДА 
by Eq. (13.8.2) Siven by Eq. (13.7.2). The chec 


/ vx 
X'X — xn (UX 
x(S) 


ec UXX = п) я [o6] 


(13.8.2) 


COMPUTING THE CORRELATION MATRIX 299 


It should be noted that the first factor on the left of Eq. (13.8.1) is simply 
the transpose of the matrix in Fig. 13.2.1. The right-hand factor on the left 
of Eq. (13.8.1) is simply the data matrix to which has been added a row of 
negative column sums divided by N. F igure 13.8.1 is a numerical example 
of the computations indicated in Eqs. (13.8.1) and (13.8.2). 


E TEE 
4 7 8 5 | A (QUIT NP 10 —4 8 
6 3 a 3 ЕТ Fer g R 
SL д ро Б а суре eo; aD 
I3; 11 93.315 | B0]| 524 -—16 20 12 0 44 
t rn dt 
10 =й 6 
m ec 


П 1 1]og| 6 —2 40| = 12 0-44] 


12 0 44 |<—check 
Fig. 13.8.1 


13.9 The Diagonal Multiplier 


The computations 

Whether we use Eqs. (13.3.2) and (13.3.3) or Eqs. (13.7.2) and (13.7.3) as 
a basis for computing the correlation matrix, the next set of operations is the 
Same. We shall therefore use Eqs. (13.7.2) and (13.7.3) to indicate the 
method. The first step is to compute a vector of the square roots of the 
diagonal elements of the matrix K. Let us assume that we have the matrix 
K together with its row sums, as indicated in Eq. (13.7.9). We shall not be 
Concerned with the last row of the right side of Eq. (13.7.9), so we merely 
indicate the first matrix row by 

[K | K1] 


Fic. 13.9.1 


We now calculate a vector of the square roots of the diagonal elements of 
and place it three columns to the left of the matrix К, as in Fig. 13.9.2. 


jh 2 3 
[002] | K 
Fic. 13.9.2 


Now in column 3 to the left of К, we write а vector of the reciprocals of the 
elements in the first column of Fig. 13.9.2. This is a vector of the reciprocals 
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of the square roots of the diagonal elements of K and is illustrated in Fig. 
13.9.3. 


1 2 3 
[D| — |Dg1| K | Kl] 
Fic. 13.9.3 


The computational checks 


Next we multiply each element in column 3 of Fig. 13.9.3 by the ret 
responding diagonal elements from K and put the entry in column 2 0 
Fig. 13.9.3. This is illustrated by Fig. 13.9.4. 


[0121 | ркрк | рк! | K | КІ) 
Fic. 13.9.4 


Now if the computations in columns 1, 2, and 3 of Fig. 13.9.4 have all 


been correct, then corresponding elements in columns 1 and 2 should be 
equal. This is true because 


DDR” = DO“ = руз (13.9.1) 


Therefore by means of columns 1, 2, and 3 we have obtained in column 3 the 
elements of the diagonal matrix by which K is pre- and postmultiplied A 


Eq. (13.7.3). These elements have also been checked, as we have noticed in 
column 2. 


13.10 Premultiplication by the Diagonal Matrix 


The computations 


First we premultiply the matrix K › augmented by a vector of row wt 
by Dx'?. Figure 13.9.4 indicates the column headings for à worksheet м 
convenient form for this multiplication. Each row of the K matrix [aa 
gether with its sums, is preceded now by the value by which each of 1 


elements is to be multiplied. We then calculate a new matrix, as 10 pe 
(13.10.1). 


DK | K1] = [ратк | ракі) (3.10.1) 
The computational checks 
The check on the multiplication is given by 


[DRK | Dgz"*(K1) | 
1 2 


1 
L|. [DRK] (13.10) 
0 3 
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The right side of this equation is a column vector of row sums of the first 
element in the first factor of the left side of the equation or the row sums of 
K after having been premultiplied by Юк!?. 'These row sums in step 3 
should be equal to the elements in step 2. Figure 13.10.1 is a numerical 
example of the operations represented by columns 1, 2, and 3 of Fig. 13.9.4. 
Figure 13.10.2 is а numerical example of the operations represented by 
Eqs. (13.10.1) and (13.10.2). 


vo | үй | zs 

ie) JE Red вал 
1 
\40 


10 —4 6 12 


40 | 440 6 —2 40 | 44 
Di DkDg Рк!1 K K1 
Fie. 13.10.1 
L uy f 10 -4 6 12 
v10 10 -4 6 | 12] | Vio vio v0 A10 
9 uod „| =| % 
vé ce om = |6 ve v6 
0 0 1 6 —2 40 | 44 6 —2 40 44 
vao 440 Үү væ | 40 
Dz"? K K1 Dg^K Dg^Kl 
10 -4 6 12 ][1 a2 
vio v10 V10 ү10 v10 «———— check 
24 6 -2 о Hilel’ 
У ye үб 
$ -2 4 | 4| | 
V40 удо yao | ү0 |10 уго 
[DRK paja] = [рК]! 
Fie. 13.10.2 


13.11 Postmultiplication by the Diagonal Matrix 


The computations 
Let us indicate the new matrix given by step 1 in Eq. (13.10.2) as В. At 
the top of the B matrix we copy the elements from column 3 of Fig. 13.9.4 


to get Fig. 13.11.1. 
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ШЭ 
EX 


Fic. 13.11.1 


The next step is to postmultiply B by DR. This amounts to Wi RA A 
each element in а column of B by the corresponding element from 


vector above it in Fig. 13.11.1. This final operation gives us the correlation 
matrix as 


BDz? = т (13.11.1) 
The computational checks 


d А Hi 
There are two sets of checks for the computation in Eq. (13.11 ) 
These are 


D =f (13.11.2) 
and 


3) 
x (13.11 


ity. 

Equation (13.11.2) says that the diagonal elements of r must all ber eR у. 

Equation (13.11.3) says that т must be a symmetric matrix, or that Tij xb? 
The procedure for caleulating the correlation matrix from the H giv 


-1/2 instead 
Eq. (13.3.2) is precisely the same as for K, except that we use Dg"? inst 
of Ру, 


SUMMARY 


1. The role of the correlation matrix: 
а. The correlation matrix is f 
science data. 
b. Matrix notation is well 
procedures. 


ы joral 
undamental in the analysis of behavi 


А 1 ional 
suited to indicating efficient computati 


2. The data matrix: 


e in 
à. Row and column sums of the data matrix are calculated for us 
subsequent checking operations, 


ауе law: 
b. The row and column Sum vectors are checked by the associative 
thus, 


1X1] = [1°Х]1 
3. The minor product moment of the data matrix: 
а. The computational formulas to consider first, are 


H = N(X'X) — (xx) 
r= Dy? HDF 
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b. The minor product moment X’X is calculated with the associative 
check 
X' (X1) = (X'X) 


4. The correction matrix (X'1)(1'X) is computed with the associative 
check 


X'1(X1) = [X^1)'X)]1 


5. The difference matrix: 
à. The product moment matrix is multiplied by N with the associative 


check 
N[X'(X1) = [IN(X'X)I1 


b. With G and L defined as 
G-N(XX) L= (X'1)(1’X) 
the matrix H = G — L is computed with the distributive check 
Gl — L1 = (G — D)1 


6. Simultaneous computations for the difference matrix: 
a. The computations for H may be computed directly by 


XX | (X’X)1 | 
H = [NI | -X'1] [zx тж 


b. The check uses both the associative and distributive laws and is 
N[(X’X)1] — (X')0/X1) = [N(X'X) — (Х'1)(1'Х)ї 


7. The case when N is large: 
а. The computational formulas are 


ROAD 
K-XX- » (6) 
r = рК 


Б. The vector of means Ux is calculated with the associative check 


©. The matrix (X' 263 is calculated with the associative checks 


8) Ге) 


d. The difference matrix К is calculated with distributive checks as 
for H. 
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"€ NEA 
K may be calculated directly with associative and distributiv 
[ 
checks by 


i ix with as- 
8. The K matrix can be calculated directly from the data matrix with 
sociative and distributive checks by 


KE [Xr X K 
SIAN 

Igi | ry к 

ТА] у 1 


9. Тһе diagonal multiplier: 


а. D¥*1 and Рк] are computed. 
b. The check for РЕ] is 


Dx[Dg'*1] = ру? 


E boe check 
10. The K matrix is premultiplied by Dgr"? with the associative 


(ОК) = Dg*(K1) 
Tf. Postmultiplication b 


y the diagonal matrix: 
a. The matrix Dk 


"ЁК = B is postmultiplied by Dg"? to give 
De ВРк\? 
b. The checks for r are 


D= І r= 


EXERCISES 


In the following, assume Y is a data matrix and let 


H = NY'Y 2 yi yy 


ҮІІ 
Ку И 
N 


ҮҮ yiyy 
с= = _ 


N N? 
ctor for 
1. Let H be expressed as the product of type III vectors. Indicate the роз! 
each of the following 


prefactors, 


(8) INL Y] (Ы (М, — ут © I, Y^ v7 


2. 


3. 


4. 


5. 


1. 


4. 
5. 


COMPUTING THE CORRELATION MATRIX 305 


Let K be expressed as the product of type III vectors. Indicate the prefactor for 
each of the following postfactors. 


(a) | z ] b [ : | (с) 4 
у гә ЖЕ = 
11У de А = 


Let r be expressed as the product of type III vectors. Indicate for each of the 
following factors the other factor. 


— Y ї Yi 
(a) Р wY z (b) [55] (с) E e 


For each of Exs. 1, 2, and 3 above, which is the best computational procedure, as- 
suming N large? 


What is the relationship between the following? 


(a) Hand K — (b Наас (е) Kand C 
ANSWERS 


y' Y Y Y N ү' Y 
( —— ©; [к= 
"ы opm] © Fr 


(a) [r | (b) [vv y ] Qu = 
T Y'Y 
é i add 
wlz] e[()r m] ez 
N N 


1(2), 2(b), 3(c). 
(à) H=NK  (H-NC (0 К= МС 


Part Ill 


The Structure of a Matrix 


Chapter 14 


Orthogonal Matríces 


In the analysis of social science data an important class of matrices is in 
frequent use. We shall call it the orthogonal matrix, taking the term from 
the mathematician, but using it in a somewhat different sense. Under the 
heading of orthogonal matrices we shall distinguish a wider variety of 
matrices than is customary among mathematicians. Some matrices, closely 
related to the orthogonal matrix, are very useful in data analysis, and we 
Shall call them orthogonal also. 


14.1 Orthogonal Vectors 


Two vectors are said to be orthogonal to one another if their minor 
Product is 0. For example, if V; and V; are two vectors, then the fact that 
they are orthogonal will be expressed as in Fig. 14.1.1. 

ҮЙ» = 0 
Fig. 14.1.1 


This states that the minor product of the two vectors is 0. Examples of 
Products of orthogonal vectors are given in Fig. 14.1.2. 


1 
=1-1+1-1=0 


B 1 2| 3)/=3+3-6=0 
—8 
Fig. 14.1.2 


2 Веѓоге considering orthogonal matrices in general, you should recognize a 
Simple type of vector that plays an important role in operations with 
orthogonal matrices. This is the normal vector. 
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14.2 Normal Vectors 


A vector is called normal if the sum of the squares of its elements is equal 
tol. If Vi isa normal vector, then we can also indicate that the sum of the 
squares of its elements is equal to 1 by saying that its minor product 
moment is equal to 1. We can illustrate this relationship by considering a 
vector V, defined as normal by Fig. 14.2.1. 


ViVi21 
Fig. 14.2.1 


Examples of normal vectors are given in Fig. 14.2.2. 


Eie Y D ES e И PEE S: 
К ЕЗ ЕЕ 
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4 
10. 
Fic. 14.2.2 


Any vector may be converted into a normal vector, or as we say, normal- 
ized, by dividing it by the square root of its minor product moment. This 
we can prove very easily. If we let V be any vector, and U be that vector 
divided by the square root of its minor product moment, we can express 
this relationship between U and V by 


V 


U= Vv (14.2.1) 
We shall now prove that U in Eq. (14.2.1) is a normal vector, that is, 
UU-1 (14.2.2) 


This we do readily by substituting the right-hand side of Eq. (14.2.1) into 
the left-hand side of Eq. (1432.2) to get 


V V «wy 
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14.3 The Definition of an Orthogonal Matrix 


Our definition of an orthogonal matrix is somewhat different from that 
of the mathematicians. In our use of the term, an orthogonal matrix is 
one whose minor product moment is a diagonal matrix. If X, a vertical 
matrix, is also an orthogonal matrix, this fact can be stated as in Fig. 14.3.1. 


X'X =D 
or 
XOX D Kak 0 ce X0 
XOX. = Ds ХХ. = 0 
fuae. 
Fig. 14.3.1 


In the first equation in Fig. 14.3.1, as is customary, we use D on the right 
to indicate a diagonal matrix. The second set of equations shows that each 
column vector of X is orthogonal to every other column vector of X, that 
1s, the minor products of all pairs of distinct column vectors are 0. 

A numerical example of an orthogonal matrix is Fig. 14.3.2. 


Дк 1 
Х=|1 3 
1 -3 
A 
1 1 1 3 0 
xx -|i 3 Z 11 : f 14 
Fig. 14.3.2 


14.4 Orthonormal Matrices 
Definition 


А special type of orthogonal matrix is the orthonormal matrix. We shall 
define an orthonormal matrix as one whose minor product moment is an 
Mentity matrix. This definition of an orthonormal matrix is illustrated in 

15. 14.4.1, 


09 = 1 
ог 
QQ. =1 0.0. =0 ... 0.0. = 0 
002 = 1 Qm = 0 
i 0.0 = 1 


Fic. 14.4.1 
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In Fig. 14.4.1, the first equation is the definition of an orthonormal matrix, 
the other equations show that each column vector of Q is normal and that 
each column vector of Q is orthogonal to every other one. We see therefore 
that an orthonormal matrix is a special case of an orthogonal matrix in 
which all of the column vectors are normal. 


Constructing an orthonormal matrix from an orthogonal matrix 


You will next see that an orthonormal matrix may be readily constructed 
from any orthogonal matrix. Suppose, for example, we have the orthogonal 
matrix X such that 


ХХ =D (14.4.1) 
Let us then define the matrix Q by 
Q = XD. (14.4.2) 


We can easily prove now that Q is an orthonormal matrix. If we take the 
minor product moment of both sides of Eq. (14.4.2), we have 


Q'Q = р-х'хр-\% (14.4.3) 


Next we substitute the right side of Eq. (14.4.1) into the right side of 
Eq. (14.4.3), and we get 


Q'Q = р-рр- (14.4.4) 
Or if we apply the law of exponents to the right side of Eq. (14.4.4), we get 
Q'Q = Deusi- = po = T (14.4.5) 


Since any diagonal matrix raised to the 0 power is equal to the identity 
matrix, Eq. (14.4.6) shows that Q is orthonormal. 

It should be clear by now that in order to change an orthogonal matrix 
into an orthonormal matrix we simply normalize the column vectors of the 
orthogonal matrix by the method of Section 14.2. 


14.5 The Major Product Moment of an Orthonormal Matrix 


The major product moment of an orthonormal matrix has some interest- 
ing and useful properties. First, we may prove that it cannot be an identity 
matrix. If Qis an n X m matrix where n is greater than m, then by defini- 
tion, 

00 =1 (14.5.1) 
We shall prove that if Eq. (14.5.1) holds, the major product moment of Q 
cannot be the identity matrix, as given by the inequality 


QQ' = I (14.5.2) 
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Now, we know that the sum of the diagonal elements of the minor produet 
moment of a matrix is equal to the sum of the diagonal elements of the 
major product moment. This we proved in Section 10.5. The relationship 
18 


l'De:ol = 1'Doo'l (14.5.3) 
But by Eq. (14.5.1), we have 
грод = II = 11 = т (14.5.4) 
But from Eq. (14.5.3) and Eq. (14.5.4), we must have 
l'Dogl = т (14.5.5) 


Now according to Eq. (14.5.5), the sum of the diagonal elements of the 
major product moment of Q must be equal to m less than n, and since this 
major product moment has n elements in the diagonal, they cannot all be 1; 
Some of them must be less than 1. Therefore, we have proved the inequality 
in Eq. (14.5.2). We shall see later on that the major product moment of Q 
must in general be distinct from any diagonal matrix. But this proof we 
Shall reserve for the last section of this chapter. 

The integral positive powers of the major product of an orthonormal 
matrix have another interesting property. We recall that an orthonormal 
matrix is defined by Eq. (14.5.1). We may prove now that any positive 
Integral power of a major product moment of an orthonormal matrix is 
equal to that major product moment, that is, it is idempotent. We express 
this rule by 


(QQ’)" = QQ’ (14.5.6) 
Suppose first we let n equal 2. Then we have 
(QQ"* = (QQ')(QQ') (14.5.7) 
Because of the associative law, we have 
(007000) = Q(Q'Q)Q" (14.5.8) 
And because of Eq. (14.5.1), we have 
Q(Q'Q)Q' = QQ’ (14.5.9) 
and finally, we have 
(QQ' = QQ' (14.5.10) 
Next we let n equal 3. By the rule of exponents, we write 
(007° = (007007) (14.5.11) 


But from Eq. (14.5.10) we can rewrite this as 
(Q'E = (QQ)(QQ) (14.5.12) 


314 STRUCTURE OF A MATRIX 


and substituting from Eq. (14.5.9) into the right side of Eq. (14.5.12), 
we have : 


(QQ’)? = QQ’ (14.5.13) 


We may then continue this same procedure for any positive integral value 
of n so that we have proved Eq. (14.5.6) by induction. 


14.6 Square Orthogonal Matrices 


The product moment 


So far we have been considering orthogonal and orthonormal matrices 
that are not square. The square orthogonal matrix has interesting and 
useful properties. We shall state without proof that only one product 
moment of a square orthogonal matrix is a diagonal matrix. This statement 
applies to the general orthogonal matrix and not to the square orthogonal 
matrix that is also orthonormal. The rule can be stated symbolically by 
the equation and two inequalities given in Eq. (14.6.1). 


If XX =D) 
and Р І (14.6.1) 
then KRAD 


The numerical example in Fig. 14.6.1 illustrates the point. 


ү? Z 
X= 1 
№ + 
4 0 
Х'Х = 1|-D 
2 
927 
XxX = 4 4 
Sz g 
4 4 
Fie. 14.6.1 


Constructing a square orthonormal matrix 


It should be clear that if we can construct a rectangular orthonormal 
matrix from an orthogonal matrix, we can also construct а square ortho- 
normal matrix from a square orthogonal matrix. Let us consider the square 
orthogonal matrix X and its minor product moment D given in Fig. 14.6.1. 
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The orthonormal matrix is obtained from the orthogonal matrix as in Fig. 
14.6.2. 


Xp = 0 
Fic. 14.6.2 


But from Fig. 14.6.1, we have Fig. 14.6.3. 


4 0 

v4 igi 
D = “all es 2 Й 

0 — 0 ү 


2 
Fie. 14.6.3 
Q is therefore given by Fig. 14.6.4. 
iz 
е р oJ] |@ 
н БЕЛ 
= 1 a 
4 =|Lo 42 A. н 
n 2 2 
Fig. 14.6.4 


It; Р P 7 
ГА easy to see now that the minor product moment of Q given by Fig. 
6.4 is the identity matrix, as shown in 


та ре =e 


А 


bie ena | gee Во 
чв wile 32 
Fic. 14.6.5 


But let US see what happens if we take the major product moment of Q as 
I Fig. 1466 
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The product moments of square orthonormals 


We now state without proof a generalization from Figs. 14.6.5 and 
14.6.6. If Q is a square orthonormal matrix, both of its product moments 
are equal to the identity matrix. This rule is expressed by 


00 = 00 = І (14.6.2) 


For Q, a square orthonormal, Eq. (14.6.2) may be expressed as in Fig. 
14.6.7. 


0.0.1 = 1 0.0. ESL sys QQ., =0 


00: = 1 00. = 0 
~ 0.0. =1 
QíQi.—1 BWa.=0 ::: QOL =0 
QQ —1 ... 00. =0 
7 QS. m1 

Fic. 14.6.7 


The first set of equations in Eq. (14.6.7) shows that all column vectors of 
Q are normal and mutually orthogonal. The second set shows that all row 
vectors are normal and mutually orthogonal. 


Infinity of square orthonormals of given order 


For any given order, there exists an infinite number of square ortho- 
normal matrices. This fact is very important when we come to consider 
the products of matrices, and particularly in factor analysis. A general 
indication of the truth of the statement may be given, although this is not 
а rigorous proof. Because of Fig. 14.6.7, we know that the number of 
scalar equations necessary to define a square orthonormal matrix is 
n(n + 1)/2. The total number of elements in a square matrix is n?. The 
number of elements we can choose at will for the square orthonormal is 
equal to the total number of elements minus the number of equations 
necessary to define a square orthonormal matrix. This is given by 


Tc mn +1) Е no D (14.6.3) 


The left-hand side of Eq. (14.6.3) shows the number of elements we can 
choose at will in determining or constructing a square orthonormal of 
order n. 
Let us consider specifically а second-order orthonormal matrix. If we 
substitute 2 for n in the right-hand side of Eq. (14.6.3), we get 
2(2 — 1) 


5 1 (14.6.4) 
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"Therefore, even for a second-order square orthonormal, we can choose one 
element at will. Suppose we let a be a value lying between —1 and 1. We 
can then write а second-order orthonormal as in Eq. (14.6.5). 


a —\ї = а? 
= 14.6.5 
А-7, | (14.6.5) 
It is easy now to verify that the right side of this equation is a square 
orthonormal, as shown in Eqs. (14.6.6) and (14.6.7). 


i-is “Лә ГЇЇ 


-yl-@ a a a 0.1 
(14.6.6) 
Aa’ = | а =ү1 — а? а Eon | 
yl- а? a -yl – а а D 
(14.6.7) 


Note that if we let a be 1/2, we get the second-order square orthonormal 
Blven by Fig. 14.6.4. Note also that there is an infinite number of values 
of a lying between 1 and —1, and that any of these values will give а 
Second-order orthonormal. Therefore, we obviously have an infinite num- 
ber of square orthonormals of order 2. 


14.7 Special Types of Square Orthonormals 


The sign matrix 


You are already familiar with several very simple types of square 
orthonormal matrices. One of these is the sign matrix, which when pre- or 
Postmultiplied by its transpose gives the identity matrix. As you have 
already seen, either pre- or postmultiplication of the sign matrix by the 
transpose simply amounts to raising it to the second power. You already 

now that the sign matrix raised to any even power is the identity matrix. 


The Permutation matrix 


You have also seen that the permutation matrix pre- or postmultiplied 
Y its transpose yields the identity matrix. Therefore by definition a 
Permutation matrix is а special case of a square orthonormal matrix. 


А 
Square orthonormal from a vector 


Another type of square orthonormal matrix may be constructed by 
Means of the identity matrix and any vector V, as shown in Eq. (14.7.1). 


2yY' 
9=1- ўт (14.7.1) 


It; A . 
tis easy to prove that if V is any vector of appropriate order, then Q is a 
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square orthonormal. This means that pre- or postmultiplication by its 
transpose yields the identity matrix. We need show this only for the case 
of one of the product moments, since we can see that the right side of Eq. 
(14.7.1) is à symmetric matrix, and therefore its minor and major product 
moments are equal. Thus we can prove that the right side of Eq. (14.7.1) 
is orthonormal by squaring both sides of Eq. (14.7.1) as in Eq. (14.7.2). 


2YYy' 2Vy' 
- ER дн 08 co. 7.2 
= (: v) (r v) пей 


Expanding the right side of Eq. (14.7.2), we have 


AVV + A(VV^(VV") 


Ct = y vy 


(14.7.3) 


But notice that the last term on the right of Eq. (14.7.3) can be written as 


4(VV^)(VV) 4(V'V)VV' 4VYV' (14.7.4) 
(V'Vy 22 үр ур 
Therefore, because of Eq. (14.7.4), the last two terms on the right side of 
Eq. (14.7.3) cancel each other, and we have 


Q-2I (14.7.5) 


which proves that Q is a square orthonormal. 
An interesting special case of the type of square orthonormal given in 
Eq. (14.7.1) is when V is the unit vector as shown in Fig. 14.7.1. 


211 2 qi 
rie aet aer" 
Fic. 14.7.1 


An example in which the unit vector is of order 3 is given in Fig. 14.7.2. 


ї ж 2 
10 „2 52 
0 Т їл 2-2 -2 
И Р 4 t LES X 2 
x pos 3 8 3 

2 2 1 

0 1 м2 ж 1] 
0 ? dex E 


Fic. 14.7.2 


14.8 Products of Orthonormal Matrices 


The product of two square orthonormals 


We shall now consider some interesting properties of produets of ortho- 
normal matrices. First, we shall prove that the product of two square 
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orthonormals is a square orthonormal matrix. Suppose we have two square 
orthonormal matrices; X and Y. The fact that they are square ortho- 
normals may be indicated by 


XX'=X'X =I (14.8.1) 


and 
YY = YY + / (14.8.2) 


Next let us consider a matrix Z that is the product of the two square 
orthonormal matrices X and Y ; thus 


Z-XY (14.8.3) 
Let us now prove that Z is also а square orthonormal, or that 

ZZ'-I (14.8.4) 
and 

ZZ=1 (14.8.5) 


First we substitute the right side of Eq. (14.8.3) into the left side of Eq. 
(14.8.4), This gives us 


XYYX =I (14.8.6) 
We rewrite Eq. (14.8.6) as 
xiu = (14.8.7) 
But from Eq. (14.8.2), we can rewrite Eq. (14.8.7) as 
— (14.8.8) 


But this is the same as Eq. (14.8.1), therefore we have proved Eq. (14.8.4). 
imilarly, we substitute Eq. (14.8.3) in Eq. (14.8.5) and get 


Y'X'XY =I (14.8.9) 

Or because of Eqs. (14.8.1) and (14.8.9), we get Eq. (14.8.10), which, 
есацѕе of Eq. (14.8.2), proves Eq. (14.8.5). 

ҮҮ= І (14.8.10) 


Тһе Product of any number of square orthonormals 


Next let us prove that the product of any number of square orthonormal 


Matrices is a square orthonormal matrix. We let Xi, Xs, ... Xn be square 
orthonormal matrices of the same order, and call the product Z; thus 
= XX... Xn 14.8.11 
We let 2 = Х.Х» ( ) 
XiX; = Y (14.8.12) 
Y2X3 = Үз (14.8.13) 


Ү.1Х, = 2 (14.8.14) 
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Now since the product of the two square orthonormals is a square ortho- 
normal, we know that Y; is an orthonormal. Therefore, we also know that 
Y; in Eq. (14.8.13) is an orthonormal. In this way, we show by induction 
finally that Z in Eq. (14.8.14) is an orthonormal, or that the product of 
any number of square orthonormal matrices is a square orthonormal 
matrix. 


Powers of a major product of orthonormals 


Next let us consider products of rectangular orthonormal matrices. First 
we shall consider the case of powers of a major product of orthonormals. 
We define two orthonormals X and Y by 


KIX a] (14.8.15) 

ХХ' =I (14.8.16) 

Е (14.8.17) 

ҮҮ' = І (14.8.18) 
We then define а major product of X and Y by 

2 = ХҮ! (14.8.19) 


First we can prove that any positive integral power of the minor product 
moment of Z is idempotent. This we indicate by 


(Z'Z» = Z'Z (14.8.20) 
We substitute Eq. (14.8.19) in Eq. (14.8.20) and get 
(YX'XY* = (YX'XY?) (14.8.21) 
But because of Eq. (14.8.15) we can write Eq. (14.8.21) as 
(YY’)" = yy’ (14.8.22) 


We have already proved that any positive integral power of the major 
product moment of an orthonormal matrix is equal to that product moment; 
therefore we have proved Eq. (14.8.20). 

Next we prove that the major product moment of Z is also idempotent or 


(ZZ^» = ZZ’ (14.8.23) 
We substitute Eq. (14.8.19) in Eq. (14.8.23) to get 
(XY'YX') = (ХҮ'ҮХ”) (14.8.24) 
And from Eq. (14.8.24) we get 
(XX) = (XX?) (14.8.25) 


We have already proved Eq. (14.8.25) so Eq. (14.8.23) is proved. 
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You should notice that the minor product moment of Z is independent 
of X and the major product moment is independent of Y. In the first case 
the X matrices cancel out, and in the second the Y matrices cancel out. 


The minor product of orthonormals 


In general the minor product of two orthonormal matrices does not 
Yield a matrix with simple properties. Consider, for example, the minor 
product of the matrices Y and X, 


U= Y'X (14.8.26) 


In general, neither the major nor minor product moment of U has any 
Simple properties. Take, for example, the square of the major product 
moment: 


(UU'y = Y'XX'YY'XX'Y (14.8.27) 
The right side of Eq. (14.8.27) does not, in general, simplify. Consider also 
the square of the minor product moment: 


(U'U)? = X'YY'XX'YY'X (14.8.28) 


Here also the right side does not simplify. It is well to remember when 
You are trying to simplify equations or computational procedures that only 
11 major products of orthonormals do we get the simple relationships 
volving positive integral powers of product moments. 


Postmultiplication of a rectangular by a square orthonormal 


Next let us consider the product of a rectangular orthonormal and a 
Square orthonormal. We shall see first that if we postmultiply a rectangular 
orthonormal by a square orthonormal, the product is a rectangular ortho- 
Normal. We let a rectangular orthonormal and a square orthonormal be 
defined respectively by Eqs. (14.8.29) and (14.8.30). 


Q'Q-I QQ'zI (14.8.29) 
H'H =I HH' =I (14.8.30) 
We let 
Z-QH (14.8.31) 
We shalt prove that Z is orthonormal, as indicated in 
ZZ-I (14.8.32) 


2 Substitute the right side of Eq. (14.8.31) in the left of Eq. (14.8.32) 
get 


H'Q'QH =I (14.8.33) 


322 STRUCTURE OF A MATRIX 


Because of Eq. (14.8.29), we rewrite Eq. (14.8.33) as 
H'H-I (14.8.34) 


But Eq. (14.8.34) is the same as Eq. (14.8.30), therefore we have proved 
(14.8.32). 


Premultiplication of a rectangular by a square orthonormal 


Next we shall prove that premultiplication of a rectangular orthonormal 
by а square orthonormal gives a rectangular orthonormal. Again suppose 
the rectangular orthonormal Q and the square orthonormal K are given. 
Equation (14.8.29) indicates that Q is a rectangular orthonormal; Eq. 
(14.8.35) indicates that K is a square orthonormal. 


КК'= КК = І (14.8.35) 
We get the product of the two matrices as 
U-KQ (14.8.36) 
We wish to prove now that U is an orthonormal, as indicated in 
UU =I (14.8.37) 
Substituting Eq. (14.8.36) in Eq. (14.8.37), we get 
Q'K'KQ = І (14.8.38) 


Substituting from Eq. (14.8.35) in Eq. (14.8.38) gives Eq. (14.8.39), which 
is the same as Eq. (14.8.29), and therefore Eq. (14.8.37) is proved. 


QQ-I (14.8.39) 


14.9 Products of Matrices by Orthonormal Matrices 


The factors of a matrix 


We shall next consider the role that orthonormal matrices play in 
produets with other matrices. Let us take, for example, the product of any 
two conformable matrices X and Y as given in 


XY-Z (14.9.1) 


Now we let a be а square orthonormal matrix. Then we define two new 
matrices by 


Ха = и (14.9.2) 

ay =з (14.9.3) 
We shall prove that u postmultiplied by v is equal to Z, as indicated by 

w-Z (14.9.4) 
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First we substitute from Eqs. (14.9.2) and (14.9.3) in Eq. (14.9.4) to get 
Xaa'Y =Z (14.9.5) 

But since a is by definition a square orthonormal, we have 

аа! = I (14.9.6) 
Substituting Eq. (14.9.6) into the left side of Eq. (14.9.5), we have Eq. 
(14.9.7), which is the same as Eq. (14.9.1). 

ХҮ=7 (14.9.7) 
Therefore we have proved Eq. (14.9.4). But you will recall that an infinite 
number of square orthonormal matrices of any given order exist; therefore, 
the matrix a could have been any one of this infinite number, and we could 
have had an infinite number of pairs of different и and v matrices. There- 


fore, an infinite number of different pairs of matrices и and v exist whose 
product is Z. 


The factors of a product moment 


Next we let X be a nonhorizontal matrix and a and b two square ortho- 
normals whose order is defined by 


Y = Xa (14.9.8) 
Z-bX (14.9.9) 
From Eq. (14.9.8), we have, by postmultiplying each side by its transpose, 
YY' = Xaa'X' (14.9.10) 

But because of Eq. (14.9.6), we have from this 
yy’ = ХХ! (14.9.11) 


From Eq. (14.9.11) we see that if a nonhorizontal matrix X is postmulti- 
plied by a square orthonormal matrix, the major produet moment of the 
Product is the same as the major product moment of X. 

i In the same way, if we premultiply each side of Eq. (14.9.9) by its 
Tanspose, we get 


ZZ- X'b'bX (14.9.12) 
and since b is a square orthonormal, this may be written 
ZZ = Х'Х (14.9.13) 


From Eq. (14.9.11) we see that if a nonhorizontal matrix X is premulti- 
E. by a square orthonormal, the minor product moment of the product 

е same as X. 
eith e can also state a general rule as follows: If a symmetric matrix S is 
ег the major or minor product moment of a matrix, it is also the major 
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or minor product moment of each of an infinite number of matrices. This 
fact has important implications for defining the primary variables of a 
science. * 


Traces of product moments involving square orthonormals 


Multiplication by a square orthonormal matrix does not affect the trace of 
the product moment of a matrix. You will recall that the trace of a matrix is 
the sum of its diagonal elements. You will also recall that because of this 
definition only square matrices have traces. We shall prove that the trace 
of the product moment of a matrix is invariant with respect to multiplica- 
tion by a square orthonormal matrix. This means simply that if we either 
pre- or postmultiply a matrix by a square orthonormal matrix to get 
another matrix, the trace of either product moment of the second matrix 
will be the same as the trace of the product moment of the original matrix. 
Again assume that a is а square orthonormal matrix. 


KO y (14.9.14) 
The rule we wish to prove can now be stated by 
Dat = I'D (14.9.15) 


Because of Eq. (14.9.14), we have 
XX’ = yy! (14.9.16) 


This you will recall we have just proved in the previous paragraph. But if 
we substitute Eq. (14.9.16) in Eq. (14.9.15), we have an identity, so the 
rule is proved for the major product of matrices. 
In Section 10.5 we proved that the trace of the major product moment of 
a matrix is equal to the trace of the minor product moment; therefore 
because of Eq. (14.9.15), we can also write 
I'Dz4 = 1,1 (14.9.17) 


For the same reason, we can also write 

I'D = Dyli (14.9.18) 
If we substitute the left-hand sides of Eqs. (14.9.17) and (14.9.18) into the 
left and right sides of Eq. (14.9.15) respectively, we have 

10.1 = 10,1 (14.9.19) 


This same гше holds if we premultiply а matrix by an orthonormal 
matrix. In this case, we consider the square orthonormal matrix b, which 
satisfies 


bb = bb’ = І (14.9.20) 
*rThurstone, L.L,, Multiple Factor Analysis, University of Chicago Press, 1947. 
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Then we define the new matrix Z by 


ox =Z (14.9.21) 
Using Eq. (14.9.20) and Eq. (14.9.21), we can prove 
1р.Л = 1р, (14.9.22) 


And remembering that the trace of a major and minor product moment ате 
equal, we have finally 


I'D = I'D. (14.9.23) 


Column sums and square orthonormals 


Next suppose that we postmultiply a matrix by a square orthonormal to 
Bet а new matrix, and then take the column sums of both matrices. We 
shall prove that the sum of the squares of the sums for the first matrix is 
equal to the sum of the squares of the sums for the second matrix. Again 
let us consider the product given by Eq. (14.9.14). We recall first that a 
Matrix premultiplied by a row unit vector gives a row vector whose ele- 
ments are the sums of the elements in the corresponding columns of the 
matrix, We recall also that the sums of squares of elements can be expressed 
as the minor product moment of a vector. Therefore, what we wish to 
Prove can be stated 


(1'Х)(Х'1) = Q'Y)(Y'l) (14.9.24) 
Substituting the left-hand side of Eq. (14.9.14) in the right-hand side of 
q. (14.9.24), we get 
(1X)(X1) = (1'Ха)(а'Х'1) (14.9.25) 
p because а is a square orthonormal matrix, Eq. (14.9.25) becomes the 
identity given by 
(п'Х)(х'1) = ('X)(X'D) (14.9.26) 


Row sums and square orthonormals 

The sums of the squares of row sums are invariant under a left ortho- 
odd transformation. If we represent this left square orthonormal 
"ansformation by Eq. (14.9.21), the rule may be stated as 


Substituting from Eq. (14.9.21) in Eq. (14.9.27), we have 
(PX (X1) = Q'X')0X1) = (ИХ) (14.9.28) 


Thi : : 
his as you see, turns out to be an identity. 
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14.10 Matrices Orthogonal to One Another 


Orthogonal matrices orthogonal to one another 


We may have matrices that are orthogonal to one another in the sense 
that their minor products are zero matrices whether the matrices themselves 
are orthogonal or not. First let us consider orthogonal matrices that in 
themselves are orthogonal to one another. We may let an orthogonal 
matrix X be defined by 


X'X-D (14.10.1) 
Now let us partition X by columns into submatrices, so that 
X-—De wot ЖЫ (14.10.2) 


Substituting Eq. (14.10.2) in Eq. (14.10.1) and partitioning the diagonal 
matrix D on the right accordingly, we have Eq. (14.10.3). 


Xi Di O era 0 
xd Жы LEE E (14.10.3) 
x ИОА 
Expanding the left hand side of Eq. (14.10.3) we have Eq. (14.10.4). 
XS UN. uy OX Di О (0 
AX. ODER oo < И ze De 2-1 (14.10.4) 
inox чы е 
From Eq. (14.10.4), we have therefore Eq. (14.10.5). 
Xi%=0 ... X!X,-0 
XiX, =0 (14.10.5) 
ete. 


It should be clear then that by partitioning any orthogonal matrix by 
columns we get submatrices such that the minor product of any pair is à 


null matrix, that is, the submatrices are mutually orthogonal to one 
another. 


Rectangular matrices orthogonal to one another 


We may have two matrices neither of which are orthogonal and yet the 
two may be orthogonal to one another. Even though the matrices are not 
orthogonal, their minor product may be a null matrix. Again consider ап 
orthogonal matrix X partitioned as indicated in 

Tren ag (14.10.6) 
By definition we have 
TES (14.10.7) 
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From Eqs. (14.10.6) and (14.10.7), we get Eq. (14.10.8). 
XU. АХ] |y. 0 
| /x: ХХ? [0 Ds (140:8) 
Next consider the square matrices a and а», whose elements may take any 
values whatsoever. Then we define new matrices Y; and Y» by 
Xam = Yi (14.10.9) 
Хз = Ys (14.10.10) 
First we will see that even though the X matrices are orthogonal, the Y 
matrices are not in general so. Consider the minor product moment of Y, 
which is given by 
ҮҮ, = ХХ (14.10.11) 
Because of Eq. (14.10.8) we can rewrite Eq. (14.10.11) as 
ҮҮ = айа! (14.10.12) 
Now since the square matrix а was taken as any conformable matrix 
whatsoever, the right side of Eq. (14.10.12) is not in general a diagonal 
matrix, so that Y, is not in general an orthogonal matrix. In the same way 


m Show by Eq. (14.10.13) that Y» is not in general an orthogonal matrix, 
us, 


үү, = @&Х&Хза, = айз (14.10.13) 

However, let us now examine the minor product of Y; and Ys From Eqs. 
(14.10.9) and (14.10.10) we get 

YiY: = aXX% (14.10.14) 
b. according to Eq. (14.10.8), X1 and X; are mutually orthogonal. There- 
Оте Eq. (14.10.14) can be written as 

үңү, = 002 = 0 (14.10.15) 
We see then that Y; and Y; are orthogonal to one another even though the 
Matrices themselves are not orthogonal. 


1411 Dimensionality of Matrices Orthogonal to One Another 


Restrictions on order 


It is important to point out certain restrictions on the orders of matrices 
at are orthogonal to one another. The proof of these restrictions is 
FE. the scope of this book, but since they are fundamental to some of 
pg developments they will be stated without proof. А number of 
Ogonal matrices of equal height cannot all be mutually orthogonal if 

* sum of their widths is greater than their height. For example, if 
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we have the orthogonal matrices Хы, Xu. ... Xea, where the sub- 
scripts indicate order, then it is not possible that XpXu; = 0 for all 
distinct values of 7 and j if Za; > b. 


Major product moment of a vertical matrix 


The implications of this rule for the major product moment of a vertical 
matrix are important. No major product moment of a vertical matrix can 
be diagonal. This rule can be proved as follows: 

We let X' be any horizontal matrix partitioned as indicated by 

XC epe Xu ll. Хь) (14.11.1) 


a 


and assume the Хы, are all orthogonal matrices. The major product 
moment of Eq. (14.11.1) is given by Eq. (14.11.2). 


Du tuc Ma Mie м. Xia Xi, 
SX XU XL. inn XL. (14.112) 
Х.Х Хь, ... ХХ, 


Since X' is horizontal, Za; » b, and therefore the off-diagonal elements of 
Eq. (14.11.2) cannot all be null matrices. Since XX’ isa major product 
moment, we have proved the rule. 


SUMMARY 


1. Orthogonal vectors. Two vectors are said to be orthogonal if their 
minor product is 0. 
2. Normal vectors: 


a. A vector is said to be normal if its minor product moment is unity. 
b. Any vector V тау be transformed into а normal vector U by 


| 


U = Р” 
3. Definition of orthogonal matrices: 
а. А vertical matrix X is said to be orthogonal if its minor product 
moment is a diagonal matrix. 


b. In particular а square matrix is orthogonal if one of its product 
moments is diagonal. 


+ 


4. Orthonormal matrices: 


а. Ап orthonormal matrix is one whose minor product moment is the 
identity matrix. Thus if X is orthonormal 


MX mI 
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b. Àn orthonormal matrix may be constructed from any orthogonal 
matrix. If 
XX =D, Q = XD“? 
then Q is orthonormal. 


5. The major product moment of a vertical orthonormal matrix Q 
a. Cannot be an identity matrix, that is, if 


QQ-I QQ'zI 
b. It is an idempotent matrix, that is, 
(QQ')" = QQ’ 
for all positive integral values of m. 
6. Square orthogonal matrices: 
à. If X is a square orthogonal but not orthonormal matrix such that 


X'X = D s I, then XX’ cannot be diagonal. 
b. If Q isa square orthonormal matrix such that Q'Q = J, then also 


QQ' = І. 
с. An infinite number of square orthonormals of order m exist. 
7. Special types of square orthonormals: 
а. The sign matrix i is a square orthonormal. 
- The permutation matrix т is a square orthonormal. 


e иф =г- WE, then for any vector V, Q is a square orthonormal. 
y'v 
ы Products of orthonormal matrices: "A 
а. The product of any number of square orthonormal matrices is a 


Square orthonormal matrix. С 
b. Given the major product moment Z = XY’ of the vertical ortho- 


normal matrices X and Y. 
(1) (Z'Zy» = z'z 
(2) (ZZ")m = zz' 

©. Neither of the product moments of the minor produet of two 
Vertical orthonormal matrices has simple properties. \ 

d. Pre. or postmultiplication of a rectangular orthonormal matrix by 
a square orthonormal matrix yields a rectangular orthonormal 
matrix. 

8, 

Products of matrices: ( 

ау Z, an infinite number of pairs of factors Х,У, exist whose 
Product is Z. 

* The factors of a product moment: 
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(1) If a nonhorizontal matrix X is postmultiplied by a square 
orthonormal matrix a, the major product moment of the product 
is the same as the major product moment of X. 

(2) If а nonhorizontal matrix X is premultiplied by a square ortho- 
normal matrix b, the minor product moment of the product is 
the same as the minor product moment of X. 

(3) If а symmetric matrix 5 is either the major or minor product 
moment of а matrix, it is also the major or minor product 
moment of each of an infinite number of matrices. 


. If we pre- or postmultiply a matrix X by a square orthonormal 


matrix to get another matrix, the trace of either product moment of 
the second matrix will be the same as the trace of either product 
moment of X. Thus if a and b are orthonormal matrices, and 
Ха = Y, bX = 2, wehave tr X'X = ir XX' = tr Y'Y = ҮҮ = 
tr Z'Z = tr ZZ’. 


. If a matrix X is postmultiplied by a square orthonormal matrix a to 


get a matrix Y, the sum of the squares of the column sums of X and 
Y are equal; that is, if 


Y = Xa then (I'X)(X'1) = ('Y)(Y'1) 


. If a matrix X is premultiplied by a square orthonormal matrix b to 


get à matrix Z, the sum of the squares of row sums of X and Z are 
equal; that is, if Z — bX, then 


Q'X^)(X1) = Q'Z^)(1) 


10. Matrices orthogonal to one another: 


а. If any orthogonal matrix X is partitioned into a type III row vector, 


the submatrices are mutually orthogonal. Thus if X — [X: X 
X,] then XX ; = 0 for all distinct values of i andj. 


b. Two matrices Y; and У, may be orthogonal (Y{Y, = 0), even 


though neither is an orthogonal matrix. 


11. Dimensionality of matrices orthogonal to one another: 


a. A number orthogonal of matrices of equal height cannot be mutu- 


ally orthogonal to one another if the sum of their widths is greate” 
than their height. 


b. No major product moment of a vertical matrix can be & diagonal 


matrix. 


EXERCISES 


1. If U, and Us are orthogonal vectors, what is U{U:? 
2. (a) If U is a normal vector, what is U'U? 


(b) If W = = what is WW? 


wv 


e 


10, 


wo N 


- (a 


-(à fQ =I- 


- If f and g are square orthonormal matrices and Xf = Y, 
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. If X is vertical and X’X is diagonal, what kind of a matrix is X? 


- (a) If X’X = I, what kind of a matrix is X? 


(b) If X’X is diagonal, D, how would you construct an orthonormal matrix Q 
from it? 


- Let Q'Q = I and QQ' = I. 


(a) What do you know about the order of Q? 

(b) What is [QQ']? 

Assume X square and X/X = D. What do you know about X X? 
(à) ирг 

(b If D- I 

(c) What kind of a matrix is X if D — I? 


7 2UU' 
What is a zero-one square orthonormal matrix called? If X — I — UU , 


2 


what do you know about 
b) ХХ (ә хх (0X 


2V.V1 21 2i z = b, what is bb"? 

ve ebd e ag, Me Be 

(b) If С, С», and C; are square orthonormal, and C = C.C;Cs, what kind of A 
matrix is C? Given X'X = I, XX’ # I, Y'Y = 1, YY' # I and Z = XY, 

(с) Whatis(ZZ^)* (d) (77° 


gX = Z, what is 


(a) YY | (b) ZZ 


(с) What is the relationship among the traces of all the product moments of X, 


Y, and Z? 
2VY' Yi 
Neo = xi- [£] 


(a) What is X!X;? — (b) УУ» 


» Let subscripts indicate order and let 


X-[Xs X Y = [Хоз Хо» Xi 4] Z-[Xus Xu: Xud 


W = [Иа Wel 


Which of the following cannot be diagonal matrices? 
(а) Хх (yy (77 @ ww 


ANSWERS 


‘UU, = 0 
(а) VU=1 (»WWe-l 
* Orthogonal 
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> 


10. 
11. 


(b) 
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Orthonormal (b) Q = XD? 

Vertical — (b) (QQ)* = QQ’ 

It is not diagonal (b XX'-I (c) Square orthonormal 
Permutation matrix (b ХХ -I (бу KX HT (9) X?=T7 


bb’ = 1 (b) Square orthonormal (c) (ZZ? = ZZ' 
(Z'Zy = Z'Z 


YY'e XX' (b) Z'Z = X'X (c) They are all equal 
XiX:i-0 (b) УУ, =0 
rY (d) WW 


Chapter 15 


The Rank of a Matríx 


The concepts discussed in previous chapters lead us now to the idea of 
Tànk, one of the fundamental properties of a matrix. As you will see later 
9n, the idea of rank is important not only as a concept in matrix algebra 
but also for its significance in scientific research. But before we define the 
Tank of a matrix, we must first consider some very simple ideas closely 
related to it. 


15.1 The Factors of a Matrix 


Infinity of pairs of factors of a product 
In scalar algebra or arithmetic it is always possible to express any number 
Ог quantity as the product of two other quantities. Figure 15.1.1 shows a 
9Y of the infinite number of ways in which you can express the number 12. 
1222x6 1223X4 


24 ‚8 12 
lec. 2 = 5 X 2 

You will remember from Section 14.9 that if a matrix can be expressed as 

a © Product of a given pair of matrices, there is an infinite number of pairs 

matrices that will yield that product. In fact, we can think of any 

atrix as the product of two other matrices. A matrix т, with n rows 

n ? columns, can always be expressed as the product of another n by m 

matrix Postmultiplied by an m by m matrix. This statement may be 
expressed as 

Xam = Yen (15.1.1) 


W] А 
here the subscripts all indieate the order of the matrices. 


Matrices as major products of factors 

There аге some matrices that can be expressed as the product of two 

exc сев Whose common order is less than the width of the matrix. For 
ample, suppose we have the product indicated by Fig. 15.1.1. 


333 


m 
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p m m 
n r Z = n 
x X 
Y Zim Хам 
Fig. 15.1.1 


Here the matrix Xam can be expressed as a major product of two other 
matrices Y ,, and Z,,, with common order т less than m. 

We know then that if a matrix Х„„ can be expressed as the product of 
two matrices whose common order т is less than m, then an infinite number 
of pairs of matrices exist with common order r whose product is Xam 
You must keep in mind, however, that only some n by m matrices can be 
expressed as the major product of two matrices. A matrix X,, cannot 
in general be expressed as the product of two matrices with common order 
less than m. 

The concept of rank is based on this highly important fact: any matrix 
may be expressed as the product of two matrices whose common order i$ 
not greater than the smaller dimension of the matrix. 


15.2 The Rank of a Matrix 


First let us assume that the matrix X „m can be expressed as the product 
of two matrices whose common order is т. But assume also that the matrix 
Х „„ cannot be expressed as the product of any pair of matrices with а 
common order less than r. Then the rank of the matrix X „m may be defined 
as т. From this definition, it is clear that the rank of a matrix is never 
greater than its smaller dimension. 

An example of а matrix of rank 1 is Fig. 15.2.1. 


3 6 3 12 

4|2 1 4=| 8 4 16 

6 12 6 24 
Ета. 15.2.1 


You see that the matrix on the right can be expressed as the product of two 
matrices (vectors) whose common order is 1. But since 1 is the smallest 
order any nonzero matrix can have, you know that the matrix cannot be 
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expressed as the product of matrices with а common order less than 1. 
"Therefore, the rank of the matrix is 1. The rule is then that the major 
product of any two vectors is a matrix of rank 1. 

To summarize, the rank of a matrix is the smallest common order among 
all pairs of matrices whose product is the matrix. 


15.3 The Rank of a Matrix with 0 Rows or Columns 


We can prove rather easily now that the rank of a matrix cannot be 
greater than the number of columns or rows that do not consist of 0 ele- 
ments. We shall prove this rule for 0 columns. If we prove it for columns, 
it can be proved for rows simply by taking the transpose, for the 0 rows of 
the matrix would then become columns in the transpose. 

Suppose we have a matrix ХӮ in which certain of the columns consist 
entirely of 0 elements. By interchanging columns, we can get another 
Matrix with all 0 columns at the right and all non-0 columns at the left. 
This we can do by postmultiplying X by a permutation matrix. We can 
xpress this operation by 

Nee = War Onil (15.3.1) 

Here Ya is a submatrix made up of the r поп-0 columns from Xam, and 

One is a null matrix with t columns, where r + = m. Now the right side 

of this equation can be expressed as the product of two matrices of common 
order т, as in 

[Yar Ond = YU 04] (15.3.2) 


If we substitute the right side of Eq. (15.3.2) in the right of Bq. (15.3.1), 
We have 


Xamr = Voller Ori (15.3.3) 
Next we postmultiply both sides of Eq. (15.3.3) by the transpose of т to get 
Х „тт! = Ут 0„]т' (15.8.4) 


Pw We know that the product moment of à permutation matrix is the 
entity matrix, or 


an =I (15.3.5) 
We let 
[Ir Orel’ = 2 (15.3.6) 
ai Zm is simply a matrix obtained by interposing 0 columns between 
Thain 


Subst: columns of the identity matrix. — — s À 
Stituting Eq. (15.3.5) and Eq. (15.3.6) in Eq. (15.3.4) gives 


Xam = Yum (15.3.7) 
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So you see that according to Eq. (15.3.7), the rank of X,» cannot be 
greater than the common order r of the matrices on the right, which is the 
same as the number of non-0 columns in Хам. A numerical example is 
given in Fig. 15.3.1. 


3040 
5020 Sui 
Xu=|2 010 Warren 
7-0) 310 iud 
6020 
3040 3 4 о, 
50225795 5:59: [10.0 
fw-[2 8 lg поа 0 0 
a A PE ees 2.53. | 6 0 
6020 6 22") |. 10.30 
x T [Ys 05] 
or 
3 4 
xe- |: ires] 
soi 1 0 0 
6 2 
Ys Uo» 0] 
or 
3 4 
59 Lo og 
ха =з 9 | кс 1 o 
т 3/L9 1 1 0 off0 10 0 
6 2 0001 
Ys Um 05] т” 
or 
3 4 3040 
5 2 5020 
Zuma 1000 lela ot o 
7 3 7030 
6 2 6.0 39 
Ys Zu = Nee 


Fig. 15.3.1 


By taking the transpose of the matrix, you can show that the rank of 2 
matrix cannot be greater than the number of its nonzero rows. 
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15.4 The Basic Matrix 


There is a certain class of matrices to which mathematicians have given 
no distinguishing name. Since these matrices play a very important role 
in the analysis of social science data, we coin the term basic matrix. A 
basic matrix is one whose rank is equal to its width or smaller dimension. 

According to this definition, any square matrix is basic if its rank is 
equal to its order. A square basic matrix is called a nonsingular matrix by 
the mathematicians, 

Most data matrices are basic. As a matter of fact, it would be the 
Sheerest coincidence if you found a data matrix of rank less than its width. 
One of the fundamental problems of science is to find data matrices that 
can be closely approximated by matrices that are not basic. Usually the 
Problem is to find predictor attributes for a prediction data matrix such 
that very slight changes in the criterion attribute vector will yield a non- 

asic matrix. It is beyond the scope of this book to show the relationship 
between the rank of a matrix and techniques of scientific prediction. 

OWever, the reduction of basic data matrices to nonbasic matrices is 
fundamental in the kinds of prediction discussed in Chapter 1. 


15.5 Types of Basic Matrices 


Besides data matrices, which are basic except for rare exceptions, there 
are other important types of matrices that are always basic. In this section 
we Shall merely list some of the most important types of basic matrices 
Without Proving that they are basic. 


Orthogonal matrices 

All orthogonal matrices are basic whether rectangular or square. Since 
orthonormal matrices are special cases of orthogonal matrices, all ortho- 
normal matrices are also basic, whether rectangular or square. Since the 
Permutation matrix is a special case of a square orthonormal, all permuta- 


i ; 
Оп matrices are basic. 


Diagonal matrices 

All diagonal matrices having all nonvanishing diagonal elements are 

asic. Since the scalar, the sign, and the identity matrix are all special 
cases of diagonal matrices with all nonzero diagonals, these are also basic. 


Tri 
"angular matrices 


All Partial triangular matrices are basic. Since the triangular matrix is 


а Special case of a partial triangular, all triangular matrices are also basic. 


1 
56 The Rank of a Supermatrix 


gt Tank of a supermatrix is defined as the rank of the simple form of 
° Matrix. We shall show that the rank of a supermatrix cannot be less 
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than the rank of the submatrix of highest rank. Let us indicate with sub- 
scripts the orders of the elements in the supermatrix M as in Eq. (15.6.1). 


Мы Ma 2 Ma 
М = Mou My. m My. (15.6.1) 
Ми My 2 M 


We can assume without loss of generality that Mau is the submatrix of 
largest rank, and that this rank is r. We can express the rank of Mau by 
Mau = LarLru (15.6.2) 
where the L's are basic matrices. 
Now if Eq. (15.6.2) is true, it is possible for a supermatrix M to exist 
which is the major product of type III vectors whose simple common 
dimension is r. In this case, we could have Eq. (15.6.3). 


Lar Mo Men ws Mae 
Lor B die wea Mus Mu 2 Mig (15.6.3) 
Ly, My, My ... My, 


However, it is not possible to find type III vectors with common dimension 
less than r whose major product will yield M, for then Mau would have to 
be of rank less than r, also. "Therefore, our rule is proved. The proof would 
be similar if we had assumed any other element to be of highest rank. 


15.7 The Rank of Products of Matrices 


The maximum rank of a product 


The rank of the product of two or more matrices cannot be greater than 
the rank of the factor of lowest rank. If we show why this is true for three 
factors, you should be able to see why it is also true for two factors or more 
than three. Let us represent the product of three factors by 


G = KaYa (15.7.1) 


where the letter subscripts indicate the order of the matrices. We assume 
that neither Х nor Zee has a rank of less than that of Ys. We assume that 
the rank of У». isr. This means, of course, that r is equal to or less than b 
or c, whichever is the smaller. But if the rank of Y, is r, then two basic 
matrices Us and V, exist, such that 


Ys = UV, (15.7.2) 


Equation (15.7.2) is a way of indicating the rank of a matrix. We can зау 
that Yə can be expressed as the product of two basic matrices whose 
common order is т. This tells us at once that the rank of Vises 
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Now we substitute Eq. (15.7.2) in Eq. (15.7.1), and get 
G = (XaUv)(Ve«Ze) (15.7.3) 


Next we define the two products in parentheses on the right of Eq. (15.7.3) 
by Eqs. (15.7.4) and (15.7.5) respectively. 


Хъ0ь = Gar (15.7.4) 
йге = Bre (15.7.5) 

Substituting Eqs. (15.7.4) and (15.7.5) in Eq. (15.7.3) gives 
G = af. (15.7.6) 


Remember we assumed that r was not greater than the rank of either 
Xa or Zee. Therefore, it cannot be greater than either a or e, and the rank 
9f G cannot be greater than r. It could, of course, be less. 


The minimum rank of a product 


Let us next consider what is the minimum rank that the product of any 
two matrices can have. Without attempting to prove the rule now, we 
Shall simply state that the rank of a product of two matrices cannot be less 
than the sum of their ranks less their common order. For example, suppose 
we have two matrices X n: and Yim of rank r and s respectively. We indicate 
their product by 

ХьыҮш = Zam (15.7.7) 


Then, According to our rule, the rank of Zam cannot be less than r + s — t. 

Th Y definition, of course, the rank of a product cannot be less than 0. 
erefore, if r + s — ( is a quantity less than 0, the rank of the product 

eee. For example, if r = 3,s=2andi=6, then r +s—t= —1 

nd the smallest possible rank for Znm is 0. M 

S you would guess, if the rank of the product of two matrices is 0, the 

Product is a null matrix. Another way of saying the same thing is that the 

Actors are orthogonal to one another. 


1 
58 Ranks of Special Kinds of Products 


rom the rules for the maximum and minimum ranks of products of 
atrices, let us work out some rules for the ranks of certain important 


Ypes of products. 


Maximum rank of product of two basic matrices 


First let us consider the product of any two basic matrices. It is easy to 
®ve that the rank of the product of two basic matrices cannot be greater 


Dr 
t 5 
an the smallest of the three dimensions. (This is true because the rank 


340 STRUCTURE OF A MATRIX 


of a basic matrix is equal to its smaller dimension, and the rank of a product 
is not greater than the rank of the factor of smallest rank.) If we have the 
product of 


XaYs = Lae (15.8.1) 


then the rank of Z cannot be greater than the smallest of the three dimen- 
sions a, b, and c. If a is smallest, then the product is a horizontal matrix 
and the rank cannot be greater than its height as shown in Fig. 15.8.1. 


b c c 
Y = 


Fig. 15.8.1 
If b is smallest, then the dimensions of the product do not indicate the 


maximum possible rank, as shown in Fig. 15.8.2. This is a major product 
moment of basic matrices, which automatically defines the rank. 


b " 2 
і | > F 
x Z 


Fic. 15.8.2 


If c is smallest, then the maximum rank is simply the width of the product; 
as shown in Fig. 15.8.3. 


b с с 
а b| Y a 
X = 7 


Fie. 15.8.3 
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Minimum rank of product of two basic matrices 

Next let us determine the minimum rank of the product of two basic 
matrices. First remember that the rank of a basic matrix is its smaller 
dimension. Then recall that the rank of a product cannot be less than the 
sum of the ranks less their common dimension. You see then that the rank 
of the product of two basic matrices cannot be less than the sum of the 
smaller dimensions of each less their common dimension. 


Minimum rank of major product of two basic matrices 

Let us see how this rule works out for the major product of two basic 
matrices. This is the product of a vertical matrix postmultiplied by a 
horizontal matrix. Such a product is illustrated by Fig. 15.8.4. 


b с с 
9 b Y = a 
X Z 


Fig. 15.8.4 


Since b is the smaller dimension of both X and Y and since it is also the 
common dimension, then the rank of Z cannot be less than b 4- b — b = b. 
Or to put it otherwise, the rank of the major product of two basic matrices 
Cannot be less than their common dimension. Another way of defining the 
tank of a matrix is by the common order of a major product of basic 


atrices that yield the matrix. 


Minimum rank of the minor product of two basic matrices 

Я The rule for the minor product of two basic matrices may be developed 
Much the same way. Figure 15.8.5 is an example of the minor product 
two matrices, 


b c e 
rf 


Fic. 15.8.5 
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You notice that the smaller dimensions of the two factors are their 
distinct dimensions, so according to the rule, the rank of the product 
could not be less than a + c — b. Therefore, the rank of the minor product 
of two basic matrices cannot be less than the sum of their distinct dimen- 
sions less their common dimension. 

You will recall now that in a special case of the minor product of two 
matrices, the factors are two submatrices obtained by partitioning a square 
orthonormal matrix between two rows or two columns. For example, 
suppose we have the orthonormal matrix Qu partitioned as indicated in 
Fig. 15.8.6. 


Fic. 15.8.6 


Here we see that a +c =b. We can write the product of Q» and its 
transpose as in Eq. (15.8.2). 


QuQu = [^ | to. Qe] = i nd (15.8.2) 
where Qu, = Qja and Qa = Qi. From Eq. (15.8.2) we have 
Qao = 0 (15.8.3) 


According to the rule, the rank of a minor product cannot be less than 
a+c—b. But since a + c = b, in this case the rank cannot be less than 
b — b, or 0. We see then that the minor product of basic factors has 9 


minimum rank when those factors are any two submatrices defining 9? 
square orthonormal. 


Minimum rank for product of basic vertical matrices 


Next let us consider the minimum rank of the product of two vertical 
basic matrices. Figure 15.8.7 illustrates such a product. 


b с с 
а b Ax = 
E | E 


Fie. 15.8.7 
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In this example the smaller orders of the two factors are b and c respec- 
tively. The minimum rank is b +c — b = c. Therefore, the minimum 
rank of the product of two basic vertical matrices is the width of the 
product, or, what amounts to the same thing, the width of the postfactor. 
Therefore the product must be basic. 


Minimum rank for product of basic horizontal matrices 


In the same way you can also discover the rule for the minimum rank of 
the product of two basic horizontal matrices, as illustrated in Fig. 15.8.8. 


Fig. 15.8.8 


Here you see that the smaller dimensions of the two matrices X and Y are 

ine b respectively. Therefore the rank of Zisa +b — b = a. Obviously, 

1 €n the rank of the product of two basic horizontal matrices cannot be 

“ss than the height of the product, or, what amounts to the same thing, 
© height of the prefactor. 


1 ; " 
29 Ranks of Products Involving Square Basic Matrices 


Basi х 
asic matrix as a factor 


Next let us see what happens to the maximum and minimum ranks if we 
чу апу basic matrix by a basic square matrix. First we shall consider 
Ec of the maximum rank. But we may either pre- or postmultiply а 
15 gar matrix by a square matrix. If we premultiply, we have Fig. 


Fig. 15.9.1 
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If we postmultiply by а square matrix, we have Fig. 15.9.2. 


b b b 
X Y 


Fic. 15.9.2 


But notice that in either case the rank of the product cannot exceed b, 
since this is the smaller dimension in both products. Therefore, the rank 
of the product of any basic matrix by any basic square matrix cannot be 
greater than the smaller order of the matrix. 

Let us now refer to Fig. 15.9.1 to determine the minimum rank of such а 
product. The sum of the smaller dimensions less the common dimension 
isa +b — а = b. Therefore, if a vertical basic matrix is premultiplied by 
a square basic matrix, the rank of the product cannot be less than its 
width, which is the same as the width of the original matrix. 

In the same way, we see by referring to Fig. 15.9.2 what the minimum 
rank of the product is if we postmultiply a basic vertical matrix by & basic 
square matrix. According to our rule this minimum rank is b + b — b-b 
Therefore, if a basic vertical matrix is postmultiplied by a basic square, 
the rank of the product cannot be less than its width or the width of the 
original matrix. 

We see, therefore, that if a vertical basic matrix is either pre- or post- 
multiplied by a basic square matrix, the rank of the product can be neither 
greater nor less than the width of the original matrix. By using the same 
logic we can show that if a horizontal basic matrix is pre- or postmultiplied 
by а basic square matrix, the rank of the product can be neither greater 
nor less than the height of the original matrix. 

Since а square matrix is а special case of а rectangular matrix, We also 
see that the product of two square basic matrices is a basic matrix. Or IR 
general, the product of any number of square basic matrices is а basic 
matrix. 


Any matrix as a factor 


Next let us see what happens to the rank if any matrix is multiplied by # 
basic square matrix. First we consider postmultiplication by a basic square. 
Suppose we have a matrix Xnm whose rank r, not less than m, is defined by 


Xam = XX m (15.9.1) 
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We let Xmm be a basic square matrix. Then from Eq. (15.9.1) we have 
XomXmm = XarXemX mm (15.9.2) 

Now we let 
XimXam = Vem (15.9.3) 


Since both X,, and Xmm are basic, Y, is also basic, for we'know that the 
product of any basic matrix by a square basic is basic. From Eqs. (15.9.2) 
and (15.9.3), we have 

Ann. = Dowel rm (15.9.4) 


We know that ris less than m. Therefore the right side of Eq. (15.9.4) is the 
major product of two basic matrices, and its rank is the common dimension 
Orr. Therefore, the rank of the left side of Eq. (15.9.4) is also r. We know 
then that the product of a vertical matrix of rank r postmultiplied by a 
Square basic matrix is also of rank r. 
Next let us premultiply X,» by a basic square Xan. We get from Eq. 
(15.9.1) 
XanXnn = X naX nArm (15.9.5) 
We let 


XnnXnr = Zar (15.9.6) 


We know that Z,, is basic because it is the product of а basic matrix by а 
Square basic matrix. We substitute Eq. (15.9.6) in Eq. (15.9.5) to get 


Х„„Х»« = ZnrXrm (15.9.7) 


As before, we know that the rank of the product on the right is exactly r. 
erefore premultiplication of any vertical matrix of rank r yields a matrix 
Whose rank is also r. | | 
We see then that pre- or postmultiplication of any vertical matrix of rank 
rhy a basic square matrix yields a product whose rank isr. Simply by con- 
Sidering the transposes of the products and remembering that square 
Matrices are special cases of rectangular matrices, we can state the general 
Tule. Pre- or postmultiplication of any matrix of rank r by a basic square 


Matrix yields a matrix whose rank is r. 


15.10 Ranks of Product Moment Matrices 


You may wonder by this time how the rank of the product moment of а 
Matrix is related to the rank of the matrix. Without attempting to prove 
a h Tule now, we shall simply state it. The rank of the product moment, 

ether major or minor of any matrix is the same as the rank of the original 

atrix. Prom this rule it is clear that the rank of the minor product mo- 
ent of a basic matrix is equal to its order. 
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Also the rank of the major product moment is simply the width of the 
original matrix. It is clear then that the major product moment of a non- 
square matrix cannot be basic. 


15.11 Rank of a Sum of Matrices 


Maximum rank for two matrices 
We shall first consider the maximum rank possible for the sum of two 
matrices. We define a matrix U, by 
ХҮ; = (л (15.11.1) 


where X; and У, are basic matrices and the common dimension on the left of 


Eq. (15.11.1) defines the rank of U;. In the same way, we define the rank of 
U: by 


MY = U, (15.11.2) 
From Emqs. (15.11.1) and (15.11.2), we get 
Ui + U: = ХА + XsY; (15.11.3) 


But the right side of Eq. (15.11.3) can be written as the minor product of 
type III supervectors as in Eq. (15.11.4). 


U, + U: = [Х, Xj [5] (15.11.4) 
2 


But the rank of Eq. (15.11.4) can be no greater than the common dimension 
of the factors on the right. And this common dimension is the sum of the 
widths of X; and X;, which is by definition the sum of the ranks of Ui and 
Us. Figure 15.11.1 will help to illustrate this point. We assume that the 
rank of {Л is b and that of U; is e. 


c b т 
ad 
U, Xi 
с е с 
а = е Y; 
Ur 


Fig. 15.11.1 (Cont. on next page) 
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b c e c 
Ui + О, =a b Y: CX 
e Y: 
Xi XS t 
с 
= a 
Ui + U: 


Fig. 15.11 1 


Maximum rank for any number of matrices 


Next we can show in very much the same manner the maximum rank of 
the sum of any number of matrices, knowing the rank of each. We define 
the ranks of n matrices U;, U», . . . Un all of the same order, by Eq. (15.11.5). 


Ui = X Yi 
U: = Хз (15.11.5) 
Us e duri 

The ranks ri, r, ... ra are the widths of the matrices Xi, Xs, ... X, re- 


SPectively, Then from Eq. (15.11.5) we have 


Chih Us enact UST ЖИЙ PRY) T uod XS. 
oe M: Es (15.11.6) 


But the right side of Eq. (15.11.6) can be expressed as the minor product of 


type ПІ Supervectors, as in Eq. (15.11.7). 
Yi 
Y: 

Mp. cat Dem ed. (15.11.7) 
Y 


The Common dimension on the right here is the sum of the widths of the X 
сев or the ranks of the U’s. Therefore the rank of a sum of any number 


Matrices cannot exceed the sum of their ranks. 
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Minimum ranks 


To find the minimum rank of the sum of two matrices, we begin by con- 
sidering two matrices ш and Уша whose ranks are defined by Eqs. (15.11.8) 
and (15.11.9) respectively. 


Usa = UU ea (15.11.8) 
Им = VacVea (15.11.9) 

Then we can write 
Uaa + Vaa = О.О + Va Via (15.11.10) 


But this can be written on the left as the minor product of two type III 
supervectors as in Eq. (15.11.11). 


Usa 
Usa + Vaa = [Us У) | — (15.11.11) 
Va 
For the purpose of our proof we assume that b is less than с. Next we let 
© Xatre = [Us Vad] (15.11.12) 
and 
Ura 
Ysea = | — (15.11.13) 
Vea 
Then substituting Eqs. (15.11.12) and (15.11.13) in Eq. (15.11.11), 
Оа + Vaa = Xatt) Y + (15.11.14) 


Now we know that the rank of a supermatrix cannot be less than the rank of 
its submatrix of highest rank. Therefore, the ranks of X and Y cannot be 
less than c, since both Уш and V.a are basic. Matrices could exist, however, 
with ranks as low as с. Taking this minimum case, then, the rank of Eq. 
(15.11.14) cannot be less than c + c — (b +c) = c — b. Therefore, the 


rank of a sum of two matrices cannot be less than the absolute difference of 
their ranks. 


SUMMARY 


1. The factors of а matrix: 
a. Any matrix can be expressed as the product of two matrices whose 
common order is not greater than the smaller order of the matrix. 
This may be done in terms of an infinite number of pairs of such 
factors. 
b. Some matrices may be expressed as the major product of two ma- 
trices. This may be done in terms of an infinite number of pairs of 
such major products. 


e 


10, 
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- The rank of a matrix is the smallest common order among all pairs of 


matrices whose product is the matrix. 


- The rank of a matrix cannot be greater than the number of columns or 


rows which do not consist of 0 elements. 


- Basic matrices: 


а. Definition: A basic matrix is one whose rank is equal to its smaller 
dimension. 
b. Data matrices are usually basic. 


- Types of basic matrices: The following types of matrices are always 


basic: 

а. Orthogonal; rectangular, and square. 

b. Orthonormal; rectangular, square, and permutation. 
€. Diagonal; general, scalar, identity, and sign. 

d. Triangular; square and partial triangular. 


- The rank of a supermatrix cannot be less than the rank of its submatrix 


of highest rank. 


- The rank of products of matrices: 


a. The rank of the product of two or more matrices cannot be greater 
than the rank of the factor of lowest rank. 
b. The rank of the product of two matrices cannot be less than the sum 


of their ranks less their common order. 


- Ranks of special kinds of products: 
a 


- The rank of a product of two basic matrices cannot be greater than 
the smallest of the three dimensions. The maximum possible rank 
(1) For a horizontal minor product is the height of the prefactor. 
(2) For a major product is the common order. 

(3) Fora product of vertical matrices is the width of the postfactor. 
b. The rank of a product of two basic matrices cannot be less than the 
sum of the smaller dimensions of each less their common order. 
€. The rank of a major product of basic matrices cannot be less than 
their common order. 
d. The rank of a minor product of two basic matrices cannot be less 
than the sum of their distinct dimensions less their common order. 
е. The product of two vertical basic matrices is basic. 
f. The product of two horizontal basic matrices is basic. 


: Ranks of products involving square basic matrices: 


a. If any basic matrix is pre« or postmultiplied by a square basic matrix, 


the product is basic. "T : 
- If a matrix of rank r is pre- or postmultiplied by a square basic 


Matrix, the product is of rank r. 


Rank of a product moment: 
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a. The rank of a major or minor product moment of a matrix is the rank 
of the matrix. | 
b. The major product moment of a vertical matrix cannot be basic. 


. Rank of a sum of matrices: 


а. Therank of a sum of any number of matrices cannot be greater than 
the sum of their ranks. 

b. The rank of the sum of two matrices cannot be less than the absolute 
value of the difference between their ranks. 


EXERCISES 


. Assume that, X can be expressed as the major product of two factors Y and Z 


whose common order is F, but that neither of these two factors can be expressed 
as the product of matrices with common order less than F. 

(а) What kind of a matrix is Y? 

(b) What kind of a matrix is Z? 

(c) What is the rank of X? 


. Express in terms of basic factors 


ION DR. 

а {2461 (5 
зво Rat. 
4 16 8 


(c) What is the rank of (a)? What is the rank of (b)? 


. Given 


Х = 


-ANG 
ooco 


6 
4 
8 
1 


(a) Express X as the product of basic factors where one of the factors is a zero- 
one matrix. 


(b) What is the rank of X? 


. Assume that all matrices in the following products are basic and let the sub- 


scripts refer to their orders. Which products must be basic? 
(а) XaX» (b) ХХ (с) Х.Х, (d) Х.Х (е) Х»Хз 
(f) Х.Х (8) XaXa (Х.Х (i) ХХ (G) XXa 


. What are the ranks of the products in Ex. 4 which cannot be basic? 


. What are the maximum and minimum possible ranks for the products in Ex. 4 


whose ranks are not known? 


. Let prescripts indicate rank in the following 


(a) iX +X +:X @Ф)„Х+ьХ+„Х (е) AX aX 
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What is the maximum possible rank for each of these sums? 

8. Let prescripts indicate rank in the following 
(à айча aty —(QiX—.X W= 
What is the minimum possible rank for each of the above? 


9. Given the supermatrix 


What is the lowest possible rank of X if Л 
(a) The rank of X,, is 3 and the rank of the other elements is 2? 
(b) The rank of Xs is 3 and that of the other elements is 4? 


10. Let X be a vertical matrix of rank r. 
(a) What is the rank of X'X? 
(b) Of XX’? 
(c) If X is basic, what kind of matrix is X'X? 
(d) If X is vertical and basic, what do you know about the rank of XX’? 


0100 

зоо 43232 0001 

Х=|2 4 0 y'=/0 121 6 ™=11 900 

143 03059 50 0010 
010 
72 5/0 0 1 
100 


What are the ranks of 
(а) YX (b) zi Y (с) т„үХ (d) Y'Y (e) TRX (f) тьХҮ' 
(8) ҮҮ (улут, 


12. Which of the products in Ex. 11 аге not basic? 


ANSWERS 


l. (a) basic (b) basie (е) F 


5, i 2 
@)2}n 23 @]2|m 4 2 (oe (Don 
3 4 
3. (a) з 6 
жы!? á ЕР) ) two 
4 воот 
A 


. Ranks are all 3 
‚ (g) (h) 
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‚ (а), (c), (d), (е), (8), @ 
(03, (h) 2 
(f) maximum 2, minimum 1, (i) maximum 3, minimum 0 
(a) 6 (b atb+c (c) 44- b 
(а) 1 (b) 0 (c) 1 (d) 0 
. (a) 3 (b) 4 
. (а) т (b) r (c) basic (d) The rank is less than the order 


Chapter 16 


Fíndíng the Rank of a Matríx 


16.1 The Triangular Factors of a Matrix 


We have discussed in some detail in Chapter 15 how the ranks of sums 
and products of matrices are affected by the ranks of individual matrices. 
We shall now consider how the rank of any particular matrix of numbers 
тау be determined. This can be done in a number of different ways. The 
Опе we shall use is computationally simple, and clearly accomplishes its 
Purpose. We shall show that any matrix pre- and postmultiplied by suitable 
Permutation matrices can be expressed as the major product of triangular- 
type matrices, This means that the prefactor is of the lower triangular 
form, and the postfactor the upper. We recall that the matrix multiplied 

Y the permutation matrices has the same rank as the original matrix, be- 
cause permutation matrices are square and basic. Now if the permuted 
Matrix is expressed as the major product of triangular-type factors, the 
common order of these factors is the rank of the matrix. This is because 
triangular-type factors are basic, and the rank of a matrix is defined by the 
common order of the major product of factors that equal the matrix. 


162 A Special Case of a Basic Matrix 


The reduction procedure 


Let us first consider a case in which we make special assumptions as in- 
‘cated below. Suppose we have a matrix X. Then let us consider the 
Product 


p= Хаб, (16.2.1) 
А Xu 


E 
vhere y 11 is assumed different from 0. 
OW the first row of iL will be 


Do XuXi, XU (16.2.2) 


= i 
FU Xu 
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The first column of iL will be 
Wa = SS = Xa (16.2.3) 
Therefore the first row of ,L will be the first row of X and the first column of 
1L will be the first column of X. We then subtract iL from X to get 
Y=X- WL (16.2.4) 


But since the first row and column of 1L are the same as the first row and 
column of X, the first row and column of Y must be null vectors. Next 
consider the product 


Lb = $ (16.2.5) 


where Y» is assumed different from 0. 
Now the first row of ¿L is 


Li, = OY; _ 9 (16.2.6) 
Y» 
And the first column of +L is 
bi 50-0 (16.2.7) 
Similarly the second row and columns of ¿L are given respectively by 
‚ҮҮ; 
214. = mw = YE (16.2.8) 
and 
2L. = Tate =Y, (16.2.9) 
2 


You see therefore that the first two rows and columns of ;L are the same 
as the first two rows and columns of У. We then subtract 2L from Y to get 


Y -iL-Z (16.2.10) 


Then since the first two rows and columns of Y and 4L are the same, Е. 
(16.2.10) shows that the first two rows and columns of Z are null vectors- 
This process is continued until the number of reductions is equal to the 
smaller dimension of X. 


Constructing the triangular factors 


Now suppose we use a slightly different notation so that the construction 
of the triangular factors becomes more obvious. We let 1X be the origina 
matrix, and as before iL is the major product of the first column and row 
vectors of ;X divided by the first row and column element. Then 


iX — 1 = 2X (16.2.11) 


where, as we have seen, the first row and column elements of „Х are all 0. 
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Then ;L is the major product of the second column and row vectors of X 
divided by the second row and column element. Then 


2X = 2L = 3X (16.2.12) 
As we have seen in the case of Eq. (16.2.10), in ;X in (16.2.12) the first two 
Tow and column elements are all 0. Then in general we have 

iX — ib = ых (16.2.13) 
Here ;L is the major product of the ith column and row vectors of ;X 
divided by the ith row and column element of ;X. The elements of the first 


* rows and columns of ;,4X are all 0. 
We next consider the definitions in Eqs. (16.2.14). 


d as DAI 
а 
ЖА 
adir? (16.2.14) 
-Vt 
L2 UV 
а; 


Here U; is the vector from the ith column of iX, Viisthe vector from the ith 

Tow of X ; and a; is the element from the ith row and column of ;X (a; is as- 
Sumed to be different from 0). 

Yow since the first row and column of »X are null, the first element of 

both U, and V, must be 0. Similarly since the first 2-1 elements of ;X are 0, 

e first i-1 elements of U; and Ví must also be 0. If we consider Eqs. 

(16.2.11), (16.2.12), and (16.2.13), we have, dropping the prescript from X, 


Xa. — L — 2b —... — ib = iX (16.2.15) 


But if 1X is nonhorizontal and of order n. X т, then 7 cannot be greater than 
^, for ifm = i then ІХ would be null, since its m rows and columns would 


all be 0, Therefore, we can write from Eq. (16.2.15) 
Xam = w+ ie uus als (16. 2.16) 
or Substituting from Eq. (16.2.14) into Eq. (16.2.16), 
z 
QUE, UVP, ОУ (16.2.17) 
Po es dU T. dn 


But Since the right side of this equation is the sum of major products of 


v 
®ctors, we can write it as Eq. (16.2.18). 
Vi 
NNUS NN zl Yi (16.2.18) 
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But the right side of this can be written as Eq. (16.2.19). 


Aq 0 
a А ү; 
0 — 0 Vi 
A sex [Ut ius es ы] ағ ? | (16.2.19) 
$4 V. 
JE RS 
Am 


Now remember that the first element of U2 is 0, the first two of U; are 0, and 
soon. Therefore, the matrix made up of the U vectors is à partial triangular 
matrix. Also the matrix made up of the v row vectors on the extreme right 
of Eq. (16.2.19) is an upper triangular matrix. Equation (16.2.19) can be 
written more compactly as 


em Da Mm (16.2.20) 


We have proved then for a special case that the matrix ,X nm can be ex- 
pressed as the product of a lower-triangular-type matrix postmultiplied by 
an upper triangular matrix. Whether a triangular-type matrix is pre- 0T 
postmultiplied by а diagonal matrix, the product is still triangular. 


16.3 A Special Case of a Nonbasic Matrix 


Going back to Eq. (16.2.15), it is quite possible that we may get 9 
matrix :1Х on the right which vanishes completely before ї = m, that 18, 
before the number of cycles through which we go is equal to the width of the 
matrix. In this case, the right and left matrices would be partial triangular. 
Such a case is illustrated in Fig. 16.3.1. 


т T m 


1X nm 


Fic. 16.3.1 


But if a matrix can be expressed as the major product of two partial triangu 
lar matrices, the common order of the two is the rank of the matrix. 
now have, therefore, a method of finding the rank of a matrix. The value 9 
i for which 1X becomes a null matrix is the rank of the matrix. & 
Let us now look carefully at the right side of Eq. (16.2.20). Notice th 
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the diagonal matrix is raised to the —1 power, or in other words, it is an 
inverse. Suppose that one or more of the a's in D, were 0. Then in Ру! the 
corresponding elements would be infinite. In this case, Eq. (16.2.20) would 
not make sense. We therefore consider the general case of the matrix pre- 
and postmultiplied by permutation matrices. 


16.4 The Product of a General Matrix and Permutation Matrices 


The reduction procedure 


Suppose that in Eq. (16.2.1), Xu is 0. Then iL is infinite, and our cal- 
culations are stopped. We need not, however, start with the first row and 
first column. Let us look for the largest element in the matrix; we assume 
this is found in the ath row and bth column. Then we calculate the residual 
Matrix as 


y =х-^әХ= (16.4.1) 


This gives all 0's in the bth column and the ath row of Y. We next look 
for the element with the largest absolute value in Y. Let this be in the cth 
Tow and dth column. Then we calculate a second residual matrix as 


z= y- rate (16.4.2) 


In the residual matrix Z, we have all 0’s in the ath and cth rows and in the 
th and dth columns. This procedure we continue until we get a null 
residual matrix. If X is a basic matrix, then the number of residual matrices 
18 equal to the width of the matrix. But if the matrix is of smaller rank, 
en the number of residual matrices, including the final null matrix, is 
“dual only to the rank of X. 


The Permutation matrices 


Now if we base our procedure on, the largest element in each residual 
Matrix, we are able to express X as the sum of major vector products just 
In Eq, (16.2.17). However, when these vectors are arranged in matrix 
orm as in Eq. (16.2.19), the matrices on the right and left of the right side of 
ae (16.2.19) are not in triangular form. It is possible, however, to get 
& ese matrices in triangular form as follows: For the original matrix and 

Ach residual matrix, we can designate the row and column number of its 


"Best element, This is illustrated in Fig. 16.4.1. 
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Fig. 16.4.1 


This shows that the largest element in the original matrix is row 6, column 2. 
The largest element in the first residual matrix is in row 4, column 3. The 
largest element in the second residual matrix is in row 5, column 5. Similar- 
ly, the largest elements for the third and fourth residual matrices are in row 
2, column 1, and row 3, column 4, respectively. We may now premultiply X 
by a permutation matrix to interchange the rows so that the numbers are 
in sequence in the first five rows. This we illustrate in Fig. 16.4.2. 


Fig. 16.4.2 


- n А 
The permutation matrix that will effect this interchange of rows let E 
call ті. Next we postmultiply X by a permutation matrix to interchang 
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the columns, so that the numbers in Fig. 16.4.2 are in sequence from left to 
right. This we illustrate in Fig. 16.4.3. 


Fig. 16.4.8 


The Permutation that will effect this interchange of columns let us call тр. 
Suppose now that we had first set up the matrix as 

т.Хтк = W (16.4.3) 

Simply by an interchange of rows and columns. Then the rule of selecting 

е largest element in each residual matrix would have resulted in factors of 

triangular type. We can say, therefore, that it is possible to find two permu- 
ation matrices rz and rpg for any matrix X such that 

EX cam = D (16.4.4) 

Where the Т?з are of triangular type and the rank of the matrix X nm is the 


со; . $ 
mmon dimension, r, of Tne and T». : 
€ are now ready to consider the actual computational procedure. 


16.5 Computational Procedure 
Given the matrix whose rank we wish to determine, we begin by cal- 
culating the row sum vector and the column sum vector. Both vectors are 
shes Summed to see that they yield the same total. These four sets of 
ations are shown in Fig. 16.5.1. 
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The reduction vectors 


Next we should find the largest element in the matrix. For convenience in 
explanation, however, we shall take the element in the first row and column. 
We first calculate the row vector, Х{ / Хи, although we could just as well 
calculate the column vector, X ;/ Xi. We must be sure to carry the sum B 
the right of the vector for our check. This step is shown in Fig. 16.5.2. 

[ХХ | Ха X10] (XFX) 
(2 3) | 5] _ 
| 2 5| = [L 1.5 | <2:5]2:5 


Fic. 16.5.2 


Notice that each element of the vector is divided by 2 including the sum 
element. The number 2.5 at the right of the bracket is the sum of the ele- 


ments, 1, 1.5, and shows that the division of each of the elements, 2 and 3; 
by 2 is correct. 


The major vector product 


The next step in the first cycle is to get the major product of the vectors 
аз indicated in Fig. 16.5.3. 


Xa Xi) | ХХ (Kal Xa X1) 
2 2 з |5 5 
LIE 1512.5] = |1 L5|25 2.5 
2 2,35 5 


Fic. 16.5.3 
Notice that the check column on the extreme right in Fig. 16.5.3 is the sum 
of row elements to the left of the partition. 


The residual matrix 


The final step in the first cycle is to subtract the major vector product 
from X as indicated in Fig. 16.5.4. 


[х= ХХЫХ) X1— X4[X3 Q0] — [X — X (XRX) 


2 8|5 2 3 |5 оо о 0 
1 3|4|—-]|1 1.52.5 |= |0 1.5 |1.5 1.5 
2 1/3 2 8 |5 0 -2|-2 —2 


Fic. 16.5.4 
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16.6 The Second Residual Matrix 


The reduction vectors 


Since we have here only a 3 X 2 matrix, it is clear that the second residual 
matrix must be 0 and we can write directly 


all. i.d (16.6.1) 


Yoo 


Y 


or 


0 
o 0 Ei L5] pro ] 
0 1.5 |- = =|0 0 (16.6.2) 


When X is of larger order, the first step in getting the second residual matrix 
18 to get the row vector. This is the second row of Y, the first residual 
matrix, divided by the second row and column element of Y. The first 


residual matrix, you recall, is 
Y =X — Х.(ХиХ\) (16.6.3) 
and 
Yi=0 and Yi =0' (11.6.4) 
First, then, we divide each element of Y}, and also the scalar Ys, 1 by Үш as 
Shown in Fig. (16.6.1). 
[YzYL|Yg(QYn0)] — (Yg Yi) 
Fic. 16.6.1 
gain remember that the scalar on the right in Fig. 16.6.1 is the sum of the 
е sents to the left of the partition, and that this checks the division of all 
“ments by y, 
The Major vector product 
The Second step is to get the major vector product as shown in Fig. 16.6.2. 
A { ^ 
[У (ҮҮ) | Y4Yz(Y0] — DYOYm Y) 
Fic. 16.6.2 


H i Е 
е again the row sum check is given by the vector on the right. The last 
P is to subtract the major vector product from the first residual matrix to 
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get the second residual matrix. This set of computations is indicated in 
Fig. 16.6.3. 


(Y = YYgY2)]| (¥1 — Y4Ya(Y;)] [= Y (Ye Ys.) 
Fic. 16.6.3 


The residual matrix 
For the second residual matrix, then, we have 
Z-Y-YaxYgY:) (16.6.5) 
and 
Zi-Z232-0 2 = 2. = 0 (16.6.6) 


so that now the first two rows and columns of the residual matrix consist of 
zero elements. 


16.7 Successive Residual Matrices 


The calculation of the third residual matrix is carried out just as in the 
case of the first two. For the third residual matrix, however, we get all zero 
elements in the first three rows and three columns. We continue to calculate 
residual matrices until we get a null residual. If the width of the X matrix 
is m and we get a null residual before the mth one, we know that the rank of 
X isless than m. For example, if X isa 7 by 5 matrix and the third residua 
matrix turns out to consist of zero elements, the rank of X is 3. ; 

You may find, for certain matrices, that if you try to proceed systemati- 
cally by reducing from the left to right and top to bottom as we have 8087 
gested, you have to divide by very small or nearly zero numbers. This 
causes trouble in that a great deal of decimal inaccuracy is apt to be intro" 
duced. You can, however, always base each reduction on the highest ele- 
ment in the preceding residual matrix and still follow the computation? 
procedure outlined. The only difference is in the rows and columns where 
the 0’s appear. Your checks work exactly the same. You must be careful w 
see that the 0’s in your major vector products correspond to those in the 
residual matrix from which it is to be subtracted. 


SUMMARY 


1. The triangular factors of a matrix: 

a. Any matrix, pre- and postmultiplied by suitable permutatio? 
matrices, may be expressed as the major product of triangular-tyP° 
matrices. 

b. The common order of the triangular-type factors is the rank of the 
matrix. 


Or 
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- Special case of a basic matrix: 


a. Any matrix X may be reduced a row and column at a time by opera- 
tions of the type 


„ХХі 
P a= о 
Y X Xa 
provided 
X470 


b. This leads to the expression of X as a product of triangular-type 
factors thus 


Kan = TonDa Tam 


where D, is diagonal. 


3. In the special case of a nonbasic matrix both factors are partial triangular. 


4. The product of a general matrix and permutation matrices: 


a. The reductions are of the type 
X Xo. 
Ca Reg. 


where Ху is the largest element in the matrix or in a residual matrix. 
- By appropriate selection of permutation matrices тр and тк, we can 
always find the triangular factors given by 


1 
тьХһътв = Tum 


Where r is the rank of X nm. 


` The computations for finding the rank of а matrix proceed through a 


Series of steps with appropriate checks. First the residual matrix is 


X „Хг. 
y x = Ec 


Calculated where X, is the element in X of largest absolute value. 


à Second, the residual matrix 
Yake 
BeY-u 


is calculated where У.а is the element of largest absolute value in Y. 


Та, the process is continued until the residual matrix is null. 


Chapter 17 


The Basíc Structure of a Matríx 


17.1 The Basic Structure of a Matrix Defined 


This chapter is concerned with a very useful concept, characteristic of all 
matrices, which we call the basic structure. Again, this term is not in general 
use by mathematicians, but it is fundamental to many of the analyses ue 
conduct with behavioral science data and helps to make clear the relation- 
ships of matrices to one another. In order to define the basic structure of à 
matrix, let us first take any nonhorizontal matrix X having n rows and m 
columns as in Fig. 17.1.1. 


X=n 


Fie. 17.1.1 


Now let us consider three other matrices as shown in Fig. 17.1.2. 


r r m 
n r r Q' 
P 


Fic. 17.1.2 


2 ы У Е 5 
In Fig. 17.1.2, the first matrix P is a nonhorizontal orthonormal with n pn 
and 7 columns. It has the same height as X in Fig. 17.1.1. For the ti 
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being, all we know about r is that it is equal to or less than m, the width of 
matrix X. The second matrix in Fig. 17.1.2 is an r X r diagonal matrix A. 
The third matrix in F ig. 17.1.2 is Q’, a nonvertical orthonormal matrix, with 
r rows and m columns. We are now ready to state the general rule that any 
nonhorizontal matrix X can always be expressed as the product of a non- 
horizontal orthonormal matrix P by a diagonal matrix A by the transpose of 
а nonvertical orthonormal matrix Q, as illustrated in Fig. 17.1.3. 


m r r m 
^ = on T r 
X Pp 
Ета. 17.1.3 


The matrices on the right of Fig. 17.1.3, you will notice, have the same 
™ensions as those in Fig. 17.1.2. : 
е shall call P the left orthonormal of X, A the basic diagonal of x , and 
“ the right orthonormal of X. Then the basie structure of a matrix con- 
Sists of its right and left orthonormals and its basic diagonal. Because of the 
finition of orthonormal matrices, you remember that 


РР = 1, (17.1.1) 
and 


Q'Q = In (17.1.2) 


However, we must remember that only if r is equal to m can the major 
Bis uct moment of Q be an identity matrix, and only if n, r and m are all 
‘Wal can the major product moment of P be an identity matrix. — 

© shall not attempt to give a rigorous proof of the rule that a matrix can 
Ys be expressed as in Fig. 17.1.3. We shall merely appeal to a well- 
Rown rule in the theory of equations. If we have n unknowns to be deter- 
Riis then precisely n independent restrictions on these unknowns must be 


Spec; А 
Pecified Іп order for us to solve for them. 


alwa, 


17 А 
2 Proof of the Basic Structure of a Basic Matrix 


a the matrix is а nonhorizontal basic matrix, then we can prove that it 
hy: be expressed as the product of a nonhorizontal orthonormal multiplied 
а diagonal and by a square orthonormal as shown in Fig. 17.2.1. 
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т т т m 
ү = mn т т 
AE P 


Fig. 17.2.1 


In this Fig., you see that we let т of Fig. 17.1.2 be equal to m. Figure 17.2.1 
can be expressed more compactly as 


X = PAQ' (17.2.1) 


The number of unknowns 


Let us see now how many unknowns we have in Eq. (17.2.1). We assume 
all of the values in X to be known and all of the elements in the factors ОЛ 
the right of Eq. (17.2.1) to be unknown. "Therefore, the number of un- 
knowns is simply the sum of the number of unknowns in each of the factors 
on the right. To indicate the total number of unknowns, we list them 2 
take the total as in Fig. 17.2.2. 


P has nXm unknowns 


Q has т х т unknowns 


A has m unknowns 
total m(n + m + 1) unknowns 
Fic. 17.2.2 


The number of restrictions 


Now let us see how many restrictions we have in terms of scalar equation? 
(17.1.1), (17.1.2) and (17.2.1) as given by Fig. 17.2.3. 


equation 1 has mety scalar equations 
” 2 » m(m + 1) ” » 
” 3 ” nm ” ” 
total m(n + m + 1) scalar equations 
Fie. 17.2.3 


) with 


Therefore, comparing the total number of unknowns in Fig. (17.2.2 o ate 


the total number of scalar equations in Fig. (17.2.3), we see that the tW 
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exactly equal. Thus we should have just enough equations to solve for the 
unknowns. A more adequate proof would also have to prove that the 
Solutions for the elements would all be real.* 


17.3 The Basic Structure of Any Matrix 


The basic structures of the basic factors 
We can next prove that any matrix can be expressed as the product indi- 
cated in Fig. (17.2.3), even if the matrix is not basic. First let us indicate 
the rank of X аз the product of two basic matrices with common dimension 
7 as shown in 
Nee Yr (17.3.1) 
Here y апа Z are basic matrices; therefore, we can indicate the basic struc- 
ture of y by Eq. (17.3.2), where now the order of A, is r and the smaller di- 
mensions of the Н matrices are r. 
Y, = HiAH; (17.3.2) 


Furthermore, the H matrices are orthonormal by definition. In the same 
Way, we indicate the basic structure of Zr by 


Zm = FiAzF; (17.3.3) 
Or, if we take the transpose of Eq. (17.3.3), we have 
Zm = FsAzFi (17.3.4) 


fe We substitute the right sides of Eqs. (17.3.2) and (17.3.4) in the right side 
Eq. (17.3.1), we have 
Х,„ = Н.А,НРА2Г (17.3.5) 


The basic structure of the basic square products 
Now Suppose we consider the product defined by 
M = A,H;F:Az (17.3.6) 


тоа Vill notice that the right side of this equation is taken from the four 
s dle factors of the right side of Eq. (17.3.5). Now M in Eq. (17.3.6) is a 
oe basic matrix. This we know because each of the factors on the right 
an Ч. (17.3.6) is a square basic matrix, and we learned that the product of 

Y number of square basic matrices is a square basic matrix. Therefore, 
* can write the basic structure of M in Eq. (17.3.6) as 


M = Аф (17.8.7) 


ї H L 
bros More rigorous treatment is given by Carl Eckhart and Gale Young in v Ap- 
1936 “Чоп of One Matrix by Another of Lower Rank,” Psychometrika, Vol. 1, Sept. 
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where gı and gs are square orthonormals of order r. 
Let us next substitute from Eq. (17.3.7) into Eq. (17.3.5) to get 
Xam = Hindi (17.3.8) 


The basic orthonormals 
Next let us define two matrices as 
Р = Н\\ (17.8.9) 
апа 
Q = Fg: (17.3.10) 


It is easy to prove now that P and Q are both orthonormals. We prove this 
for P and the proof for Q is exactly the same.* From Eq. (17.3.9), we have 


РР = AH (17.3.11) 
But because H, is orthonormal, Eq. (17.3.11) becomes 

P'P = gig (17.3.12) 
And since g is a square orthonormal, Eq. (17.3.12) becomes 

P'P=I (17.3.13) 


The proof that Q is orthonormal is the same as for P. 
Next we substitute equations (17.3.9) and (17.3.10) in (17.3.6) to get 


Xam = PAQ' (17.3.14) 


The right side of Eq. (17.3.12) is then in basic structure form, since P and Q 
are both orthonormals, and A is a diagonal matrix. Furthermore, the сол 
mon orders of the matrices on the right are т. Therefore, we have proved 
that any nonhorizontal matrix may be expressed as the product from left to 
right of а nonhorizontal, a diagonal, and a nonvertical orthonormal, and the 
order of the diagonal is the rank of the matrix. 


17.4 The Basic Diagonal Matrix 


Signs of the elements 


The basic diagonal plays a fundamental role in matrix algebra and in the 
analysis of social science data. We shall therefore prove some importan 
rules about its properties. First we can prove that the diagonal elements 9 
the basic diagonal can always be taken as positive. For the sake of argu" 
ment, let us just suppose that some of these elements were negative. Let i 
be a basic diagonal that just happens to have several negative dier 
elements. We multiply D by a sign matrix in which the appropr!? 
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diagonal elements are negative so that we get a diagonal matrix all of whose 
diagonal elements are positive, so that 


e A = 1р (17.4.1) 


In Eq. (17.4.1) i is the sign matrix, D is the basic diagonal with negative 
elements, and A is the basic diagonal with all positive elements. Now 
Suppose that we had expressed X as the product 


Xam = kDq' (17.4.2) 


where D is the diagonal matrix with some negative elements and k and q are 
orthonormal matrices. We could write Eq. (17.4.2) in two different ways as 


Xam = (ki) iD)q (17.4.3) 


or 
Xam = k(Di)(iq) (17.4.4) 


It is easy to see that Eqs. (17.4.3) and (17.4.4) are the same as Eq. (17.4.2) 

cause the sign matrix multiplied by itself gives the identity matrix, so 
Substituting the left side of Eq. (17.4.1) into Eqs. (17.4.3) and (17.4.4) gives 
respectively 


Xam = (ki)Ag' (17.4.5) 
and 


Xen = kA(iq') (17.4.6) 


Now notice that in Eqs. (17.4.5) and (17.4.6) the diagonal matrix A has 
Y definition all positive elements. Notice also in Eq. (17.4.5) that k post- 
multiplied by т is still an orthonormal matrix, because, as we have seen in 
‘hapter 16, an orthonormal matrix multiplied by a square orthonormal 
Sives a product that is orthonormal. You remember that the sign matrix is 
* Special case of a square orthonormal. Similarly, in Eq. (17.4.6), q is pre- 
реа by i, but this product is also orthonormal. It should be clear, 
€refore, that if in Eq. (17.4.2) certain of the elements in D are negative, 
di may be changed to positive simply by changing the signs of the ele- 
vents in the corresponding columns of k or in the corresponding rows of 0. 
на lave proved, therefore, that the elements in the basic diagonal of a 
atrix may always be chosen positive. 


O 
rder of magnitude of the elements 


a Asuming now that all the elements in A are positive, let us next assume 
àt they are all different in value; that is, we assume that no two of the 
We shall prove that the basic diagonal A may 
lements are in descending order of 
t.* Let us assume that we have a 


ia 

tye elements are equal. 

s be chosen so that its diagonal e 

"itude from upper left to lower righ 

бы! “descending order of magnitude” we mean that the Lth element of A is greater 
°F equal to the (L + 1)th element. 
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basic diagonal A, in which the elements are not in descending order of mag- 
nitude. We indicate the basic structure of Х „„ by 


Х = Р.А.0; (17.4.7) 


where P, and Qj, are basic orthonormals corresponding to As. Equation 
(17.4.7) represents some particular arrangement or ordering of the diagonal 
elements of A that are not in general in descending order of magnitude. 
Now by pre- and postmultiplication of A, by an appropriate permutation 
matrix and its transpose, we can get a diagonal matrix in which the elements 


are in descending order of magnitude. Suppose we indicate this reordering 
of the elements by 


А = Taba, (17.4.8) 
Аз a simple illustration, let us assume that A, and A are given by Fig. 
17.4.1. 
20-9 9:0: 10 
4,=|0 3 0 А=|0 2 0 
001 0 5 1 
Fig. 17.4.1 


It is easy to show that thea 


ppropriate permutation matrix to change A, to A 
is Fig. 17.4.2. 


© 1-0 
7, -|1 0 Oj=7% 
00 1 


Fic. 17.4.2 


In this particular example, т, 


is a symmetric matrix. Figure 17.4.3 shows 
the steps by which we get the 


elements of A, in descending order of magni- 
tude. 
2 0 o][fo 1 o 0. 20 
0:587 0. 12.70: | es зоо 
0^ 0 3 0-20: 1 De 
Aa T, Аат; 
О 301] [70 2 4 3.00 
10 0113 0 0|-]|0 2 0 
DAO FIOO T 0. 0-1 
Ta Аат, Tabat, 
Fic. 17.4.3 


Now Eq. (17.4.7) can be written in the form 


Xam = Р.т! naQ] (17.4.9) 
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You will notice that the permutation matrices in Eq. (17.4.9) cancel, and 
therefore Eq. (17.4.9) is the same as Eq. (17.4.7). Because of Eq. (17.4.8) 
we can rewrite Eq. (17.4.9) as 


Хал = Par ATQ, (17.4.10) 

Next let us define the two matrices P апа Q respectively as 
P = Рух, (17.4.11) 
Q = Qara (17.4.12) 


Notice that P and Q are still orthonormal matrices because they are the 
product of orthonormal matrices by square orthonormal matrices. You 
recall that a permutation matrix is a special case of a square orthonormal 
matrix. Because of Eqs. (17.4.11) and (17.4.12), we can rewrite Eq. 
(17.4.10) as 
Xam = PAQ' 

Now Eq. (17.4.13) has all the elements of A in descending order of magni- 
tude. Therefore we have proved that by an appropriate interchange of 
columns in P, and rows іп Q/, the basic diagonal of a matrix can always be 
expressed with its diagonal elements in descending order of magnitude. 

We have proved now that any nonhorizontal matrix may be expressed as 
the product from left to right of a nonhorizontal orthonormal, a diagonal, 
апа a nonvertical orthonormal, where the diagonal has all positive elements 
in descending order of magnitude, and its order is the rank of the matrix. 

You should remember that there is one and only one basic diagonal for 
апу given matrix X. Furthermore, if all of the diagonal elements of the 
basic diagonal are different from one another, there is only one left ortho- 
normal and only one right orthonormal in terms of which the basic structure 
сап be expressed. The case is somewhat different, however, if the basic 
diagonal has some elements that are not distinct from one another. 


One set of equal diagonal elements 


Let us first consider the case in which all of the elements in A are distinct 
except a certain number which are all equal one to another. We could then 
write the basic structure of X in supermatrix notation as Eq. (17.4.14). 


Х = [Pi Pj Is | [e] (17.4.14) 


This equation means that we have grouped the distinct diagonal elements of 
Ain a submatrix A and all of the equal elements in a submatrix Ау. Further- 
more, the P and Q orthonormals have been partitioned to correspond with 
the submatrices Ay and A, respectively. Now since all of the elements in A 
are equal, we can write 


A = СІ (17.4.15) 
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here C is а scalar quantity equal to the elements in A;. If now we multiply 
w 
out the right side of Eq. (17.4.14), we have 


X = PAQ, + PiAiQ; (17.4.16) 
Because of Eq. (17.4.15) we can write this as 
X = PAQ + СРО, (17.4.17) 


But notice that the second term on the right of this equation is the pn 
product of two orthonormal matrices, Suppose we select any of an infinite 
number of square orthonormals h and write 


P= P (17.4.18) 
and 

@ = Qh (17.4.19) 
From Eqs. (17.4.18) and (17.4.19) we can write 

Pix pi (17.4.20) 
and 

Qi = qii (17.4.21) 


If now we substitute the ri 


ght sides of Eqs. (17.4.2 
right side of Eq. (17.4.17), 


0) and (17.4.21) into the 


we have 
Х = PAR + Ср (17.4.22) 
But since h was defined as a square orthonormal, Eq. (17.4.22) can be 
written as 
X = Рм + Cpg, (17.4.23) 


Or, to write (17.4.23) in the form of the product of super matrices, we get 
Eq. (17.4.24). 


X = [р ^ 0о][0; 4.24) 
PoR [o A Е H^ 
d алена thit pi апа a in Eq. (17.4.24) can be obtained in an 1n- 
finite number of y Ply by sele cting a pa 


More than one set of equal diagonal elements 


In general, if we have more than one s : ic 
1 t bas 
diagonal, the general equat; et of equal elements in the 


ion correspond; жии, 
Eq. (17.4.25). ponding to Eq. (17 4 94) is given 
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Ao 0 0 A 
0 I 0 i 
X=[Po р Pal п 
0 0 &E 11 d. 


(17.4.25) 


None of the p and q submatrices on the right side of Eq. (17.4.25) are 
unique. There are as many pairs of submatrices that are not unique in the 
orthonormals as there are sets of repeated elements in the basic diagonal. 


17.5 The Basic Structure of the Product Moment Matrix 


Next we shall see what happens in the basic structure of a matrix when it 
is pre- or postmultiplied by its transpose. First let us express the matrix in 
its basic structure form, as indicated by 


X = P,AQ’ (17.5.1) 


The minor product moment 


Let us now take the minor product moment of X. We premultiply each 
side of Eq. (17.5.1) by its transpose to get 


X'X = QAP'PAQ' (17.5.2) 
But since P is an orthonormal, its minor product moment is an identity 
matrix, so Eq. (17.5.2) may be written as 
X'X = QuQ' (17.5.3) 
Equation (17.5.3) shows first that the basic diagonal of the minor product 
moment of a matrix is equal to the square of the basic diagonal of the matrix 
Second, it shows that the left orthonormal of a minor product moment is the 
transpose of the right orthonormal of the matrix. And third, it shows that 
the right orthonormal of a matrix and its minor product moment are the 
same. 
The major product moment 
Next let us postmultiply each side of Eq. (17.5.1) by its transpose as in 
XX' — PAQ'QAP' (17.5.4) 
But since Q is an orthonormal, its minor product moment is an identity 
and (17.5.4) may be written as 
XX' = PXP’ (17.5.5) 


We see in Eq. (17.5.5) that the basic diagonal of the major product moment 
of a matrix is the square of the basic diagonal of the matrix. We also see 
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from Eqs. (17.5.3) and (17.5.5) that the basic diagonals of the major ү 
minor product moments of a matrix аге equal. From Eq. (17.5.5) we a : 
see that the left orthonormal of a matrix and its major product пеши 3 
the same. Finally we see that the right orthonormal of the major pro ue 
moment is equal to the transpose of the left orthonormal of the matrix. 
To summarize, we may say then that first, the basic diagonal of either 
product moment of a matrix is the square of the basic diagonal of the 
matrix; second, that the orthonormals of the major product moment are 
from the left orthonormal of the matrix; and third, the orthonormals of the 
minor product moment are from the right orthonormal of the matrix. 


Powers of product moments 


Next let us see what happens to the 


basic structure when we raise the 
product moment of a matrix toa 


power. First let us square both sides of 
Eq. (17.5.5). This gives 
(XY. = (Олд (Q.A:Q!) (17.5.6) 
We let 
ð=4 (17.5.7) 


Remembering that the minor product moment of Q is the identity matrix, 
and substituting from Eq. (17.5.7) in Eq. (17.5.6), we have 
(X'X} = Qoi! 


From Eq. (17.5.8) we see that the orthonormals of the square of the minor 
product moment are the Same as the orthonormals of the minor product 


moment itself. The basic diagonal of the minor product moment is squared 
to get the basic diagonal of the 


s Square of the minor product moment. 
now we multiply both sides of Eq К 


- (17.5.8) by th ding sides of 
Eq. (17.5.3) and use Eq. (17.5.7), we get TS Resear 


(17.5.8) 


(X xy = Q'Q'QaQ' (17.5.9) 
Or from Eq. (17.5.9) we have 


(X'Xp = 0920), 


(17.5.10) 
By continuing in the same way, 
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minor products of orthonormals that yield the identity matrix, we can di- 
rectly write Eq. (17.5.12) corresponding to Eq. (17.5.11). 


(XX) = Рд"Р' (17.5.12) 


Equation (17.5.12) shows that the Ьазїс orthonormals of the major product 
moment raised to any integral power are the same as the basic orthonormals 
of the original major product moment. It also shows that the basic diagonal 
of the power of a product moment is obtained by raising the basic diagonal 
of the major product moment to the corresponding power. Equations 
(17.5.11) and (17.5.12) show that the basic diagonals for a given power of 
either the major or minor product moment are the same. 


17.6 Matrices with the Same Orthonormals 


The major product 
Suppose we consider two matrices a and b, which have for their basic 
Structure the same orthonormals but different basic diagonals. These we 
can illustrate by 
a — PAQ' (17.6.1) 
and 
b = PAQ' (17.6.2) 
It is obvious, of course, that if a and b as given here have the same ortho- 


normals, they must be of the same order. Let us first then consider the 
major product of the matrices a and b given by 


ab’ = PAQ'QAP; (17.6.3) 
From this we have 
ab’ = PAAP (17.6.4) 
Now let us take the transpose of this to get 
ba! = PAAP' (17.6.5) 


But notice that since diagonal matrices are commutative, the right side of 
Eqs. (17.6.4) and (17.6.5) are equal. Therefore, we can write 


ab’ = ba’ (17.6.6) 

Or to put Eq. (17.6.6) in words, we can say that if two matrices have the 

Same basic orthonormals, their major product is a symmetric matrix. 
The minor product 

Also from Eqs. (17.6.1) and (17.6.2) we can write a minor product as in 

a'b = QA,P'PAQ' (17.6.7) 
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or from Eq. (17.6.7) we have 


a'b = QAAR’ (17.6.8) 
Taking the transpose of both sides of Eq. (17.6.8), we have 
b'a = QA Q: (17.6.9) 


But the right sides of Eqs. (17.6.8) and (17.6.9) are equal, since diagonal 
matrices are commutative. Therefore, we may write 


ab = Ьа (17.6.10) 
From this it follows that the minor product 
orthonormals is a symmetric matrix. Ther 


product of two matrices having the 
matrix. 


of two matrices with equal 
efore, either the major or minor 
same orthonormals is a symmetric 


Product moment matrices with same orthonormals 
Next let us consider an 


orthonormals. The righ 
normal. We can, theref 


y two product moment matrices having the same 
t orthonormal is the transpose of the left ortho- 
ore, write two such matrices as 


S, = Qa,Q! (17.6.11) 
and 
8, = QIQ’ (17.6.12) 
Now from Eqs. (17.6.11) and (17.6.12) we can write 
8,8, = Qà.Q'Qa,Q' (17.6.13) 
and 
88, = 09,0'09,0' (17.6.14) 
Or from Eqs. (17.6.13) and (17.6.14) we have respectively 
8.8 = QIQ’ (17.6.15) 
апа 
SS, = 09,90 (17.6.16) 


But since diagonal matrices are commutative, the right sides of Eqs. 
(17.6.15) and (17.6.16) are equal; therefore, the left sides are equal, and we 
can write 


(17.6.17) 


Se = Qa! (17.6.18) 


^T 
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From Eqs. (17.6.15) and (17.6.18) we can write 


8.8.8 = Q0.0,0,Q" (17.6.19) 
and 

5.88. = Q0,010.Q" (17.6.20) 
Also from Eqs. (17.6.16) and(17.6.18) we can write 

S.S,S, = Q09.09.Q' (17.6.21) 
and 

88.8. = Q9,0,0.Q' (17.6.22) 


The right sides of Eqs. (17.6.19) through (17.6.22) are all the same except 
for the order of the basic diagonals, and since these are all commutative, the 
four are equal. Therefore, all the products on the left-hand sides of these 
equations are also equal. In the same way, we could have interposed the 
matrix S. between the other two matrices and shown that the product was 
not changed. Using the same procedure and remembering that the minor 
product moment of an othonormal matrix is the identity matrix, it is easy 
to show that all product moment matrices with the same orthonormals are 
commutative, one with another. We have also the rule that the product of 
any number of product moment matrices with the same basic orthonormal 
is equal to a matrix having the same basic orthonormal and a basic diagonal 
that is the product of the basic diagonals of the factors. 


17.7 Special Powers of a Product Moment Matrix 


Fractional powers 


Let us consider now two product moment matrices whose basic structures 
аге defined by 


Si = 090' (17.7.1) 
and 
5 = QOQ’ (17.7.2) 
Squaring both sides of Eq. (17.7.2), we get 
S; = 0д!°0'0д\°0' (17.7.3) 
But this can be written as 
S; = 09129120) (17.7.4) 
or 
S; = QaQ' (17.7.5) 


Notice that the right sides of Eqs. (17.7.1) and (17.7.5) are the same; there- 
fore, we can write 


S = 81 (17.7.6) 
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Taking the square root of both sides of Eq. (17.7.6), we get 


S: = 81° (17.7.7) 
In the same way, suppose we had let S; be defined by 
S, = Qui! (17.7.8) 
Then we could show that 
5: = 5 (17.7.9) 


is true. Or, from Eq. (17.7.9), we would have 


$, = 513 (17.7.10) 


Suppose we have the general equation for the basic structure of a product 


moment matrix, as in 

S = QoQ' (17.7.11) 

"Then the rule for the nth root of а product moment matrix can be stated as 

Su» = Qang (17.7.12) 

If a is any positive value, either fractional or integral, the general rule is 
expressed in 


Se = Qaà«Q' (17.7.13) 


The inverse 


We shall next let the 


exponent a in Eq. ( 
either positive or negati 


ve. Therefore, if a 


8^ = 09-0) 


17.7.13) be àny real number, 
takes the value —1 we have 


(17.7.14) 
Suppose now we multiply each Side of Eq. (17.7.14) by the corresponding 
side of Eq. (17.7.11) to get 


88-1 = QaQ'Qa-19’ 


(17.7.15) 
From this we have 
55-1 = Qag-1g’ (17.7.16) 
or 
55-1 = 00, (17.7.17) 


Now notice that if Q is Square, that is, if S is a basic matrix, the right side of 
Eq. (17.7.17) is an identity, so that we have 
55-1 = 7 


(17.7.18) 
In this particular case, S-! is called the inverse of S. This subject will be 
considered more fully in Chapter 19. 
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17.8 The Basic Diagonal and Traces Involving Product Moments 


The product moment 


Another rule that we shall find convenient is that the sum of the elements 
in the basic diagonal of a product moment is the same as the trace of the 
major or minor product moment itself. This we can prove rather easily by 
considering the minor product moment of X , as given by 


X'X = QaQ’ (17.8.1) 
Suppose now we define a matrix Z by 
2 = 09° (17.8.2) 
From this we can rewrite (17.8.1) as 
AUX = 22" (17.8.3) 
Then 
ir X'X = tr ZZ! (17.8.4) 
but from Chapter 10 we know that 
ir ZZ! = tr Z'Z (17.8.5) 
From (17.8.2) we have 
ir Z'Z = tr) M QrQu: (17.8.6) 
ог 
ir Z'Z = 1 6 (17.8.7) 
Substituting Eq. (17.8.7) in Eq. (17.8.5), we get 
ir ZZ' = tró (17.8.8) 
and substituting Eq. (17.8.8) in Eq. (17.8.4) gives 
tr X'X = tré (17.8.9) 


But since the trace of a diagonal matrix is simply the sum of its elements, we 
have from Eq. (17.8.9) 

tr X'X = Yol (17.8.10) 
And since the traces of а major and minor product moment are the same, we 
have 

ir XX! = l'àl (17.8.11) 


Therefore we have proved the rule. 


Trace of powers of product moments 


In the same way, we can write for any specified power of the major or 
minor product moment the relation between its trace and its basic diagonal 
by 

tr(X X^) = tr(X'X) = 1'д"1 (17.8.12) 
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therefore the general rule that the trace of any power of а major o 
Eo: у, moment is equal to the sum of the elements in the cor- 
munor n д 
responding power of the basic diagonal. 


17.9 The Product of a Matrix by a Square Orthonormal 


If any matrix is either pre- or postmultiplied by a squ 
the basic diagonal of the product is the same as for the origi 
can be shown very easily if we start with th 
matrix X as given by 


are orthonormal, 
nal matrix. This 
e basic structure form of the 


X = PAQ' (17.9.1) 
Suppose now we premultiply (17.9.1) by a Square orthonormal, say Н\, as in 
HX = HiPAQ; (17.9.2) 


On the left of the basic diagonal in this equation, we have the orthonormal P 
premultiplied by a square orthonormal, which yields a product that is ortho- 


normal. In a similar manner, we could postmultiply Eq. (17.9.1) by a 
Square orthonormal H, as in 


XH; = PAQ'H, (17.9.8) 
On the right side of the 
orthonormal multiplied 
that is orthonormal. Th 
sides are in basic structu 
it was in Eq. (17.9.1). 


basic diagonal A in Eq. ( 
by a square orthonorma 
erefore, in both Eqs. (17. 
re form and the basic di 


17.9.3), we again have an 
l, which yields a product 
9.2) and (17.9.3), the right 
agonal A is still the same as 


SUMMARY 


X = Pag’ 


= I, and A is diagonal. 

2. In a basic matrix the number of unknown elements in the basic Structure 
factors is precisely equal to the number of restrictions placed on these 
elements by the definition. 


where Р/Р = Q'Q 
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b. Therefore, any nonhorizontal matrix may be expressed as the product 
from left to right of a nonhorizontal orthonormal, a diagonal, and a 
nonvertical orthonormal matrix. 

c.. The order of its basic diagonal is the rank of a matrix. 


4. The basic diagonal: 

a. The signs of the basic diagonal elements may all be taken as positive. 

b. The basic diagonal elements may be taken in descending order of 
magnitude from upper left to lower right. 

c. The basic diagonal is unique for any given matrix. 

d. Any nonhorizontal matrix may be expressed as the product from left 
to right of a nonhorizontal orthonormal, a diagonal, and a nonvertical 
orthonormal, where the diagonal has all positive elements in descend- 
ing order of magnitude, and its order is the rank of the matrix. 

е. If the basic diagonal includes sets of equal elements, only the sub- 
matrices from the basic orthonormals not corresponding to these sets 


аге unique. 
5. The basic structure of the product moment matrix. If the basic structure 
of X is X — PAQ', then the basic structure of 
à. The minor product moment is 


X'X = QA’ 
b. The major product moment is 
XX! = PAP! 


€. The power of a product moment is 
[х'Хү = QAQ 
[XXT = PAP’ 


6. Matrices with the same orthonormals: 
a. The major and minor products of matrices with the same ortho- 


normals are symmetric matrices. 
b. Product moment matrices with the same orthonormals are commuta- 


tive with respect to multiplication. 


7. Special exponents of product moment matrices. 
a. Any fractional power of a product moment matrix is given by a 
matrix with the same orthonormals and the corresponding fractional 


power of the basic diagonal. If S = QéQ’ 
Su» = QgilQ! 
b. The inverse of a basic product moment matrix S = Q6Q' is given by 


S = Q571Q’ 
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8. The basic diagonal and traces involving product moments: 


a. The sum of the elements in the basic diagonal of a product moment is 
the same as the trace of the product moment. If X'X = QsQ' 
tr(X'X) = tr(XX^) = 1%1 


b. The trace of any power of a product moment is given by 
ir(X'X)^ = tr(XX') = 1°%"1 


9. If any matrix is either pre- or postmultiplied by a square orthonormal 
matrix, the basic diagonal of the product is the same as for the original 
matrix. 


Chapter 18 


Fíndíng the Basíc Structure of 


a Matríx 


18.1 The General Solution for the Basic Structure 


Practical significance 


You have seen in Section 17.1 that any matrix can be expressed as the 
product of two orthonormals and a diagonal matrix. This product we have 
called the basic structure of the matrix. A general area of interest centering 
around the basic structure of data matrices in psychology is actually the 
Study we have referred to previously as factor analysis. In recent years 
other disciplines, such as sociology, anthropology, political science, eco- 
nomies, and even biology, have become interested in the application of 
factor analysis techniques to matrices of experimental data. One of the 
most important elements in factor analysis is determination of the basic 
structure of a matrix. It is therefore important for us to know how the 
basic orthonormals and the basic diagonal of a data matrix may be found. 
In this chapter we shall first consider the subject theoretically and then set 
up à computational routine. 


The left orthonormal 
You will recall that the general equation for the basic structure of а 
matrix is 
X = PAQ' (18.1.1) 
Where P and Q are orthonormals, and A is a diagonal matrix. Now if we 
knew the elements of the matrix Q, it would be quite easy to solve for P and 
^. We could postmultiply both sides of Eq. (18.1.1) by Q and get 
XQ — PAQ'Q (18.1.2) 
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But since Q is orthonormal, Eq. (18.1.2) would be written on the right as 
XQ = PA (18.1.3) 


So you see that by simply multiplying X on the right by Q we would have 
on the right only P and A. We could readily solve for A by taking the minor 
product moment of both sides of Eq. (18.1.3) to get 


Q'X'XQ = АР'Рл (18.1.4) 
But since Р is also orthonormal, this could be written as 
ОХ'ХО = л? (18.1.5) 


Equation (18.1.5) shows that the left side 
the right side is diagonal; therefore b 
side, we would solve for A by 


must be a diagonal matrix, since 
y taking the square root of the left 


(ОХО) = A 


(18.1.6) 
Then to solve for P, we would 


simply postmultiply both sides of Eq. (18.1.3) 
by A^! to get 
XQA = Р (18.1.7) 
Or, if we substituted from the left of Eq. (18.1.6) into the left of Eq. (18.1.7), 
we would have 


XQ2(Q’X'XQ)-172 = Р 
If only, then, we knew the elements of Q, Eq. ( 
find the basic diagonal A and Eq. (18.1.8) would enable us to find the left 
orthonormal P. Only rarely, howev: 


ег, do we have available the right ortho- 
normal, Q, for any given data matrix, so our chief problem becomes one of 
first solving for the Q matrix. 


(18.1.8) 
18.1.6) would enable us to 


18.2 Finding the Right Orthonormal of a Data Matrix 


get column 1 of Q together with 


Then we get column 2 of Q, and the 
basic diagonal, 


diagonal. 
corresponding second element of the 


manner until all columns of Q and all elements of the basic diagonal have 
been solved for. The method assume: 

generally be the case with data e logic upon 
which the procedure is based and then show how the computations are 
carried out. 


The minor product moment 


First we get the minor product moment of both sides or Eq. 


(18.1.1) as in 
X'X = QAP'PAQ' 


(18.2.1) 
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But since P is orthonormal, the right side of Eq. (18.2.1) can be written as 
X'X = QAQ’ (18.2.2) 
You are already familiar with this equation, which shows that the basic 
diagonal of the minor product moment of a matrix is the square of the basic 
diagonal of the matrix. Also it reminds you that the left orthonormal is 
simply the transpose of the right orthonormal. You see then that merely by 
taking the minor product moment of X, we have eliminated the left 
orthonormal. 
For convenience we let. 
д = А? (18.2.3) 
Equation (18.2.2) then becomes 
X'X = 090' (18.2.4) 
You will recall that without loss of generality we may assume that the ele- 
ments in A are in descending order of magnitude from upper left to lower 
right. 
Powers of the minor product moment 
Now let us raise the minor product moment of X to some positive power 
^ where n is an integer. This gives 
(X'Xy = 0д"0' (18.2.5) 
Equation (18.2.5) is taken from Section 17.0 and shows that in raising a 
product moment to any power, the basic orthonormals remain unchanged 


and the basic diagonal is raised to the same power as that of the product 
moment. 


If n is large 
Suppose now that we let n be a very large number and that we divide Eq. 
(18.2.5) on both sides by a number K, as in 
1 7 Wee д” " 
к^ Х)" = Qx9 (18.2.6) 


Now let us look at this value K on the assumption that n is very large and 
that none of the elements in à are equal. Suppose first we consider the basic 
diagonal of Eq. (18.2.6) in expanded form, divided by the constant K as in 


Eq. (18.2.7). 


9r 
K OF 0 
97 
д" 0 
= K 
K zi (18.2.7) 
0. 4@ on 
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If we assume that K is equal to дү, we can write Eq. (18.2.8) 


= (18.2.8) 


Suppose now that the ratio of д» to д, is, let us say, .8. We could then select 
n large enough so that this ratio raised to the nth power would be 0 to any 
specified number of decimal places. For example, .8 to the 32nd power 
would be 0 in the first two decimal places. If the д” were in descending 
order of magnitude, then for this particular example all elements in Eq. 
(18.2.8) would be 0 to at least two decimal places, except the first one, which 
would be unity. Suppose, however, that K was not exactly equal to a? but 
was just roughly of that general order of magnitude. Even in this case, if we 
chose n large enough, we could have all elements but the first in Eq. (18.2.8) 


0 to any specified number of decimal places. Provided then that we choose 
K properly, and specify the degree of decimal accuracy we desire, Eq. 
(18.2.7) can be written in Eq. (18.2.9 


) to any desired degree of decimal ac- 
curacy. 
1 0 0 
Sud B 3» ow 18.2.9) 
mu Qi | 
0 0 0 


But the matrix on the ri 


ght of Eq. (18.2.9) can be written as the major 
product mom ent of an € vector, as in 


© = Жең (18.2.10) 
Substituting the right side of Eq. (18.2.10) into the right side of Eq. 
(18.2.6), we have 
1 п 
КХХ)" = aeg (18.2 .11) 


We can write Eq. (18.2.11) as 
1 n 
КХХ)" = Toeg (18.2.12) 


Next let us examine the product of the orthonormal Q by the e, vector on 
the right of Eq. (18.2.12). This is 


Qa = Qı (18.2.13) 
Equation (18.2.13) reminds us that when we postmultiply any matrix by an 
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e; vector, the product is simply the ith column vector of that matrix. If we 
substitute the right side of Eq. (18.2.13) into the right side of Eq. (18.2.12), 
we get 


gx - 20.01 (18.2.14) 


Equation (18.2.14) shows that if we raise the minor product moment of а 
matrix to a sufficiently high power and divide it by an appropriate constant, 
the resulting matrix, to a specified number of decimal places, is proportional 
to the major product moment of the first column vector of its basic ortho- 
normal. 


The arbitrary vector 


Next let us consider any arbitrary vector V that is not orthogonal to Qu. 
Suppose we postmultiply both sides of Eq. (18.2.14) by V to get 


Faxy = 2010047) (182.15) 
Because Q’, V is a scalar, Eq. (18.2.15) can be written 
gv = [5 v Je; (18.2.10) 


Equation (18.2.16) shows, therefore, that if д» is less than à;, and if the minor 
product moment is raised to a sufficiently high power and divided by an ap- 
Propriate constant, then multiplication by any vector that is not orthogonal 
to the first column of the basic orthonormal yields a vector proportional to 
the first column of the basic orthonormal. The power to which the matrix 
must be raised in order to give any specified degree of decimal accuracy de- 
pends on the ratio of the second to the first element in the basic diagonal. 
The smaller this ratio, the lower the power to which the matrix must be 
raised. 


The solution for Q.1 
From Eq. (18.2.16), we can easily solve for Qı. First we rewrite Eq. 
(18.2.16) as 


gv «Y (18.2.17) 
Then Q., will be given by 
Qi- (18.2.18) 


Since Q ; is by definition normal. 
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The solution for à 


i i dily obtained from 
lue of Q. is found, the value of à is rea 
S "as on premultiply both sides by Q’, and postmultiply by Q. 
to get | 
Q(X'X)Q.1 = 0.090". (18.2.19) 


But since Q is an orthonormal, we have 


Q'Qi-oa (18.2.20) 


Equation (18.2.20) simply says that Qı is orthogonal to each row of Q' ex- 
cept the first, which is Q^;, and therefore multiplication by Q yields unity. 
Substituting Eq. (18.2.20) in the right of Eq. (18.2.19), we have 


Q(X'X)Q, = ева (18.2.21) 

But we know that 
аде; = à (18.2.22) 
Substituting from Eq. (18.2.22) into the right of Eq. (18.2.21), we have 
QC" X)Q, = ә, (18.2.23) 


"Therefore, if we premultiply the product moment matrix by the transpose 
of the first column of its orthonormal, and postmultiply by the first column 
of the orthonormal, we get the first element of the basic diagonal. 


18.3 The Iterative Solution for the First Vector of Q 


Successive vector multiplications 


Raising a matrix even to the second power can be pretty laborious. If it 
must be raised to a hi i i 


(X'X)V, = y, (18.3.1) 
(XXV. = y, (18.3.2) 
XXW: = y, (18.3.3) 


Now notice that if we substitute the left Side of Eq. (18.3. 1) for V, in the 
left side of Eq. (18.3.2), we have 


(X'XyV, = y, (18.3.4) 
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Then if we substitute the left side of Eq. (18.3.4) for V2, in the left side of 
Eq. (18.3.3), we have 


(X'X9Vo = V; (18.3.5) 
In the same way, we could show 
(X'X) Vo = V, (18.3.6) 


Equations (18.3.1) to (18.3.6), therefore, show that if we multiply the 
original product moment matrix by an arbitrary vector to get another 
vector and continue to multiply the product moment matrix each time by 
the last vector so obtained, this amounts to multiplying the original 
arbitrary vector by some corresponding power of the product moment 
matrix. Therefore, although it would be prohibitively laborious to raise a 
matrix to a fairly high power and then multiply by an arbitrary vector, the 
same results can be achieved in a very small fraction of the time by multi- 
plying the original product moment matrix successively by the vectors ob- 
tained from the multiplications. 


The scaling constant for iterated vectors 


So far we have not considered how to obtain the value K, which we first 
used in Eq. (18.2.6). If we were simply to proceed as in Eqs. (18.3.1) 
through (18.3.4), the elements of the successive V vectors could become 
either very large or very small. Actually, therefore, we must adopt some 
procedure for keeping the size of the elements within reasonable limits. We 
therefore proceed as follows: we multiply the product moment matrix by an 
arbitrary vector Vo. In most cases the Vo vector may be a unit vector. Our 
first multiplication is 


SoVo = Ui (18.3.7) 


where to simplify our notation we use So for X'X. Instead of now multiply- 
ing So again by Ui, we first divide the elements of U, by some appropriate 
constant. It has been convenient to use as this constant the largest absolute 
value in U;.* If we indicate this largest absolute value by Uiz, then we 


define a new vector as in 


Uy _ 
т = й (18.3.8) 


The largest absolute value in V; will now be unity. Next we multiply So by 
Vias in 
SoVi = Us (18.3.9) 


*This method was first proposed by Harold Hotelling in “Analysis of a Complex of 
Statistical Variables," Journal of Educational Psychology, 1933, Vol. 24, pp. 417-441. 
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We then find the element with the largest absolute value іп U; and calculate 
а new V; vector by 
U; 


т = Из 18.3.10) 
О, Ve ( 
The general equation for the multiplication of S; by a vector is 
SV. = U; (18.3.11) 
The general equation for calculating the V vector from the U vector is 
0; є 
——- £V. 18.3.12) 
Ui, " ( 


The Q.; vector 


The process is continued until 


within the limit of decimal accuracy de- 
sired, the last V vector calculated 


is equal to the one immediately preceding, 


or until 
у= У, (18.3.13) 
If the process has been continued until this equation is satisfied, we can 
write 
St = У, = CQ, (18.3.14) 
where 
К =й... ‚Ум (18.3.15) 


In other words, K is simply the 


product of all of the largest elements from 
the U vectors. The constant C 


in Eq. (18.3.14) is given by 


С = Жолу, (18.3.16) 
Actually, however, we rarely find it necessary to calculate either K or C 
given by Eqs. (18.3.15) and (18.3.16). The vector Qı can be calculated 
directly from either the U п or V, vectors, since they are both proportional 
to one another and since we know that Q. is a normal vector. Calculating 
Q. from U, therefore, we have 


Qa = Tar (18.3.17) 


The 6; scalar 


The value for ài, the first element in the basic diagonal, is readily ob- 
tained as follows: assume that V _, is proportional to U, within the limits 


of decimal error. We know then that Va and U, are both proportional to 
Ол. Let us begin then with 


SV. = 0, (18.3.18) 
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We have said that both vectors V and U were proportional to 0) 1; therefore 
we could write 

Via = 004 (18.3.19) 


where g is а proportionality constant. If we substitute Eq. (18.3.19) in 
Eq. (18.3.18) and also use the basic structure form of So, we have 


4080'0 = Un (18.3.20) 
Or, because of Eq. (18.2.20), we can write Eq. (18.3.20) аз 
09де = Un (18.3.21) 
Because of Eqs. (18.2.23) and (18.2.24), we can write 
gàiQe = Un (18.3.22) 
Because of Eq. (18.2.13), we can write Eq. (18.3.22) as 
901 = Un (18.3.23) 
Because of Eq. (18.3.19), we can write Eq. (18.3.23) as 
1.1 = Un (18.3.24) 


But remember that according to our plan of computations, the largest 
element in each V vector is always unity. But if the largest element of 
V,.1 is unity, and U, as given in Eq. (18.3.24) is simply the V vector 
multiplied by à; then the largest element Ur; in U, is ду. Therefore, 
Eq. (18.3.24) shows that when a given U vector is proportional to the 
preceding V vector, the largest element in the U vector is the first element 
in the basic diagonal. 


18.4 Successive Q Vectors and ô Elements 


The first residual matrix 


So far we have obtained the first column vector of the right orthonormal 
of X and the first element in the basic diagonal of the product moment, 
Which, you will remember, is the square of the first element of the basic 
diagonal of X. Our next problem is to get the second vector of the ortho- 
normal and the second element of the basic diagonal. We may write the 
basic structure of So as а sum of major products of vectors, as in 


S, = 0.0 + 90:0 +... + Innn (18.4.1) 


Assuming now that we have already solved for Q ; and ð;, let us calculate a 
residual matrix given by 


Sı = So — 20.0. (18.4.2) 
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From Eqs. (18.4.1) and (18.4.2), we have 


Sı = 90:05 +... H Dn nn (18.4.3) 


We should note in passing that the rank of S; is one less than the rank of So. 
This is true because as you can see on the right of Eq. (18.4.2), the basic 
orthonormal of S; has one less column in it than the one in Eq. (18.4.1). 


The second Q vector and 6 element 


We see, therefore, that to get Sı we simply subtract from So а matrix that 


is the major product moment of Q , multiplied by à,. Having calculated Si, 


we then proceed to find О» and à; in exactly the same way as we found Q.: 


and ài. We use Eqs. (18.3.7) through (18.3.17) to calculate Q. except that 
now we use S; instead of Sy. То get д», we simply take the largest element 
in the final U vector. 


Other residual matrices, Q vectors, and 6's 


Having calculated Q, and 9, we then calculate a second residual matrix, 
as in 


8 = Sı — 90 0" (18.4.4) 
To get Q.3 and дз we proceed with S. 


? exaetly as we did with S d So. 
It should be noted, howeve: iss qune 


4 › only the Q vecto ing to a 
relatively small number of the largest, 6’s will be aa 


18.5 Finding the Left Orthonormal 


The procedure for computing 
follows: first we get the square 
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root of the basic diagonal à of the product moment matrix that gives the 
basic diagonal A of the matrix X. This is given by 


А = д!” (18.5.1) 


Next we get the reciprocal of this matrix and postmultiply Q by it. The 
product is premultiplied by X to get P. The complete set of operations for 
computing P is given by 


X(Qo-w) = P (18.5.2) 


18.6 Computations for the Basic Structure 


The product moment matrix 


In getting the basie structure of a matrix X, we begin as we did for the 
correlation matrix. We calculate the minor product moment matrix of X 
exactly as we did in calculating the correlation matrix. In these compu- 
tations, we have a sum of row and column elements as a result of our 
checking operations. We start, then, with the product moment matrix, at 
the bottom of which will be a row vector of column sums. Figure 18.6.1 
Shows the worksheet layout for getting the successive U and V vectors. 


0 = V: = U: = 
КАД E 807» 
2 
asovi | ERE | uso; 
10 А 103 
Fig. 18.6.1 


The successive U and V vectors 


In Fig. 18.6.1, the U; column is obtained by multiplying both the S, 
matrix and its sum vector by Vo. The last entry in the U, column is 
obtained merely by summing all of the U; elements, and should be equal to 
the entry immediately above it. The V; column is obtained by multiplying 
each element of U, by the reciprocal of the largest absolute value in the 
U, column. This multiplication also includes the element immediately 
below the V, vector. The final element in the column is obtained merely 
by summing the V; elements. This should equal the value immediately 
above it. The remaining U and V vectors are calculated in the same way. 
These operations continue until two successive V columns are equal. 

Ordinarily, it is sufficient to carry the computations to three decimal 
places. Actually, for the first few approximations, a smaller number of 
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imals may be carried. Sometimes it is advisable to carry only ies 
icu until two successive V columns are equal, then two arimen i 
A RO V columns are equal, and finally three decimals until two 
А V columns are equal. This procedure cuts down somewhat on 
5 
the computational labor. 


The О. vector 


The next step is to caleulate the Q vector from the final V vector. This 
we do by the operation indicated in Fig. 18.6.2. 


1 2 3 
|t See ie ae e eoo een 
Vs 
VV, грр 
f IV e 
a (1 San x v (1'Q.)NUs 


Fic. 18.6.2 


The first column in Fig. 18.6.2 is sim 
the bottom of vectors 1,2. 
facilitate explanation of wha 


ply the final V vector. The rows at 
and 3 are labeled from a to e inclusively to 
t the various steps require. у 

Ows a and b are simply the check entries from 
Fig. 18.6.1. The entry in row € of column 1 is the sum of the squares of the 
elements in the V vector. The entry in row d is the square root of the 
entry in row е. The entry i 


à Y ‹ d. 
n row e is the reciprocal of the entry in row 
ing row a is 


column 2 is the square of th 
equal to the largest absolut 
square root operation. 
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The w vector 


Column 3 above row a is obtained by multiplying every entry in column 2 
&bove and including row a by the entry in row d of column 2. The entry in 
row b of column 3 is obtained by summing the elements in column 3 above 
row a. This entry should equal the entry in row a of column 3. The entry 
in row c of column 3 is the sum of the squares of the entries in column 3 
above row a, and should equal the largest absolute value in the final U,, 
vector. This should give back the first element of the basic diagonal. 


The residual matrix 


The major product moment of the w vector in column 3 of Fig. 18.6.2 
will be equal to the major product moment of the Q. vector multiplied by 
the first element of the basic diagonal. This is the matrix we want to 
subtract from S» in order to get our first residual matrix, Sj. Figure 18.6.3 
Shows how we may perform these operations. 


a [1'$% — (1^w)w' 


Ета. 18.6.3 


We start with the product moment matrix So, which has been bordered at 
the bottom by a vector of column sums. On the right of this matrix we 
attach with a paper clip the w vector with its sum. This matrix is then 
Postmultiplied by the identity matrix, bordered at the bottom by the w 
vector in row form with the sign of each of its elements reversed. This 
Product is the first residual matrix Sı. The a row of the product is obtained 
merely by multiplying the sum vector at the bottom of the Sy matrix by 
the bordered identity matrix on the right. Row b at the bottom of the 
Product matrix is obtained by summing all of the columns above row a. 
Rows a and b should be equal. It is, of course, possible to calculate first 
the ww’ matrix and then subtract this from the S matrix. The appropriate 
associative and distributive checks should be carried out. 
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Successive Q vectors 


The operation shown in Fig. 18.6.3 gives the first residual matrix Sı 
together with the summation row at the bottom. You now proceed with 
the S; matrix in exactly the same manner as you did with the Sy matrix to 
get the second column vector Q.» of the basic orthonormal and the second 
element à; of the basic diagonal. The process is continued until you have 
the desired number of Q vectors and 6 elements. 


The left orthonormal 


The first step in getting the left orthonormal is to calculate the reciprocals 
of the square roots of the basic diagonal elements. This means that you 
find the reciprocals of the values in row d of column 2 for each worksheet 


illustrated in Fig. 18.6.2. You will, of course, have a worksheet of this kind 


for each vector of Q. These reciprocals are entered in row € of the worksheet 
in Fig. 18.6.4. 


above row b. are ready to calculate the column entries 
The entries in column 1 down thro 

n | ugh row i iplying 

the corresponding entries of column 2, fis. te аы A T 

un giis » row е. To get the entry in row b of column 1 we simply 

add the elements in column 1 above row a. The entry in row b of column 1 
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Should equal the entry in row a of column 1. The remaining columns of 
Fig. 18.6.4 are obtained in the same way using the appropriate Fig. 18.6.2 
worksheet. In this way we get a matrix of Y vectors that is equal to Q 
postmultiplied by the — 1/2 power of the basic diagonal. 

Figure 18.6.5 shows how from the data matrix X and the matrix Y 
obtained in Fig. 18.6.4 we finally get P, or the left orthonormal. Starting 
with the primary matrix X bordered at the bottom by a vector of column 
sums, we postmultiply this by the matrix Y. The a row in the product 
matrix is simply the row at the bottom of X postmultiplied by Y. The 
b row represents the column sums in the product matrix down to but not 
including the a row. The b row in the product should equal the a row. 
The product XY is the P matrix. As а final check, you should calculate 
the minor product moment of P to see that it does yield the identity 
matrix within limits of decimal accuracy. 


Ето. 18.6.5 


18.7 The Basic Structure for Standard Measures 


In most applications of basic structure analysis it is customary to find 
the Q and 5 matrices of the correlation matrix rather than the minor 
Product moment of the data matrix X. This means that instead of the 
basic structure of X we are concerned with the basic structure of a matrix 
Z of standard measures. This is a matrix obtained as follows: 


cte 163 (18.7.1) 


Z = sD} (18.7.2) 
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From Chapter 13, we know that 


"e 21 (18.7.3) 
We have then 
т = QiQ' (18.7.4) 
From Eq. (18.7.4) we find Q and à precisely as outlined for So. 
wq» = Ри) (18.7.5) 
From (18.7.5) 
-Z (08-и) p (18.7.6) 
VN 


Actually, of course, we need not calculate Z as such. We get Dz: in the 


process of calculating the correlation matrix. Therefore from Eqs. (18.7.1), 
(18.7.2), and (18.7.6), we have 


хүр 
Pa (PPE gue = 18.7.7) 
| А би ; 
We let 
d = (NDz)-i* (18.7.8) 
Y = 05-12 (18.7.9) 
Then we calculate 
dY - M (18.7.10) 
IX 
Cu oe (18.7.11) 


From Eq. (18.7.7) through Eq. (18.7.1 1) we have 


Р= ХМ -w (18.7.12) 
It can be readily shown that Eqs. (18.7.1 
much more economical than Eqs. (18.7.1) 


SUMMARY 


1. General solution for the basic Structure: 
a. The basic structure of a data ma; 


Ё 3 trix is of gr ical i rtance 
in the social and biological sciences. карашы пра 
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b. If the right orthonormal Q' of X were known, the basic diagonal 
would be 
А = (Q'z'XQ)i* 
and the left orthonormal would be 
Р = ХОА^! 


- Finding the right orthonormal of a matrix: 
a. The basic structure of the minor product moment is 


X'X = Q5Q’ 
b. The basic structure of a power of a product moment is 
[X’X]" = QQ 
с. If n is sufficiently large and K is appropriately chosen, we can write 
g OX» = 10:05 
d. If V is any vector not orthogonal to Q.; we have 
ô” 
Famy = (žo) = v 


e. The solution for Q. is 
ү: 
9: ^ Av'y 
f. The solutions for ô is 
à = Qa(X'X)Qa 


- Iterative solution for the first vector of Q: À 
à. The product of any power n of X'X = So by an arbitrary vector can 


be obtained by successive multiplication of the form 
Sia = V: = (X'X)'Vo 


b. To keep the V; vectors in hand, we have 


SyV ia = U; 
Ui 
Ke= as, 


where U;z is the element of largest absolute value in Ui. 
с. The Q. vector is calculated when Un = Un to any desired degree of 


decimal accuracy by 
Un 


Qı = DAA 
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d. The ô, element is simply U;,, the largest element іп U,. 


4. Successive Q vectors and à elements: 
а. The first residual matrix à; is given by 


8, = 8, — GOROR 


b. ©» and ô are obtained from S: as in Зс and 3d above. 
c. In general 


Se enis s òQ ;0'; 


and the solutions for Q.; and б; are as in 3c апа d above: 


5. The left orthonormal of X is given by 


P = Xo- 


6. Computations for the b 


asic structure proceed through the following 
steps: 


a. The product moment matrix Х'Х. 

b. The successive U and V vectors for 01. 
€. The Q. vector, 
d. The w. vector. 
€. The residual matrix S;. 


f. The successive Q.; and w; vectors and S; residual matrices. 


easures: 
computed. 
omputed from 
r = 080, 


€. The P matrix is computed from 


7. Basic structure for standard п 
a. The correlation matrix r is 
b. The Q and ô matrices are c 


P-XM р 
where 


М = (N Dxx)=12Q5-12 


Part |V 


Matríx Solutíons 


Chapter 19 


The Inverse of a Matríx 


19.1 The General Inverse and the Basic Structure 


You have already seen in previous chapters how we define matrix addi- 
tion, subtraction, and multiplication. You have seen that matrix addition 
and subtraction resemble very closely scalar addition and subtraction. You 
have also seen that matrix multiplication is a little more complicated than 
Scalar multiplication, which is in fact a special case of matrix multiplication. 


The inverse defined 


We may now define an operation in matrix algebra that is analogous to 
division in scalar algebra. You remember that in scalar algebra, when 
We divide a dividend by a divisor this amounts to dividing 1 by the divisor, 
or taking the reciprocal of the divisor and then multiplying the dividend 

y the reciprocal. For example, if we want to divide 3 by 2, we can first 
divide 1 by 2 to get .5 and then multiply 3 by .5. In scalar algebra, if 
We multiply a number by its reciprocal, we get 1. Therefore, we consider 
for a given matrix whether we can find another such that if the two are 
multiplied together, we get the identity matrix, which as you know, 
takes the place of 1 or unity in matrix algebra. If, for example, we have 
a matrix a and can find another matrix b such that the product of the two 
В the identity matrix, then we call b the inverse of a and а the inverse 
of b 


The general inverse and the basic structure 


In order to clarify the definition of inverses, we shall start with the basic 
Structure of the matrix. Although it is not common among mathematicians 
to treat the subject of inverses from this point of view, matrix inversion 
Will be clearer to you in terms of the basic structure. Let us start with the 
basic structure of a nonhorizontal matrix X, as given by 


X = PAQ' (19.1.1) 
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You recall that P and Q are the orthonormals of X, and A is its basic 
о 4 1 à 
diagonal. Now consider the matrix Y as given by 


Y = Pag’ (19.1.2) 


i i i 1 i .1.1) and (19.1.2) are the 
4 will notice that the right sides of Eqs. (19 ‘ 
с that the basic diagonal of Eq. (19.1.2) is the inverse of the 


basic diagonal of Eq. (19.1.1). The orthonormals of X and Y are the same. 
Next consider the transpose of Eq. (19.1.2) given by 


Y'= Qaap' (19.1.3) 


You recall that so far we have placed no restrictions on X. It need not be 
Square or basic. If it is not basic, then Q’ 

We shall now give a most unusual bu 
Matrix Y’ as given in Eq. ( 


will be a horizontal matrix. 

t extremely useful definition. 
19.1.3) will be called the general inverse of X as 
given in Eq. (19.1.1). We therefore define the general inverse of any 
matrix as the transpose of another matrix whose basic orthonormals are the 
same as the original, and whose basic diagonal is the inverse of that of the 


original. This is not the traditional mathematical definition of an pape 
of a matrix. The traditional definition, as you will see, is in fact a speci 
case of this definition. 


The major product of a matrix and its general inverse 


Let us see now what happens if we postmultiply the matrix X as given in 
Eq. (19.1.1) by its gen 


eral inverse as given in Eq. (19.1.3), as in 


XY’ = PAQ'QA-Ap! (19.1.4) 
But the right side of this e 


quation includes the minor product moment of 
an orthonormal matrix, s 


O we get 


XY’ = Paa-ip! (19.1.5) 
Or, from this equation 


, Since 
inverse, we have 


Хү’ Yx' (19.1.7) 
We also know that 


if the major product 
matrix is raised to any 


т moment of an orthonormal 
positive integra] Powe: 


T, that power is equal to the 


THE INVERSE OF A MATRIX 405 
original product moment. Therefore, if n is any positive integer, we have 
from Eq. (19.1.6) 

(IYF = XY (19.1.8) 
(EX = VX! (19.1.9) 


The minor product of a matrix and its general inverse 


Next let us see what happens if we premultiply the matrix X by its 
general inverse. From Eqs. (19.1.1) and (19.1.3) we have 


Y'X = QA (P'P)aQ* (19.1.10) 
Or from Eq. (19.1.10) we get 
Y'X = QAAR (19.1.11) 
And from Eq. (19.1.11) we get 
Y'X = QQ’ (19.1.12) 


We see then from Eq. (19.1.12) that premultiplication of a matrix by its 
general inverse yields the major product moment of its right orthonormal. 
Postmultiplication, as you saw by Eq. (19.1.6), yielded the major product 
moment of the left orthonormal. We see therefore, also, that if we raise the 
Product of X premultiplied by the general inverse to any positive integral 
Power, this power will be the same as the original product. This rule is 
given by 

(Y'X)' = Y'X (19.1.13) 


We should note in passing that the general inverse of the transpose of а 
matrix is equal to the transpose of the general inverse. This you can easily 
See from Eqs. (19.1.1), (19.1.2), and (19.1.3). If you take the transpose of 
Eq. (19.1.1), then the right side of Eq. (19.1.2) is the general inverse of this 
transpose. This, of course, is the transpose of Eq. (19.1.3). 


The general inverse of a basic matrix 

So far we have made no assumptions about the rank of the matrix X. Let 
US assume now that X isa basic matrix. If X is a basic vertical matrix, 
then its right orthonormal is square. But either product moment of a square 
orthonormal is the identity matrix, as you already know. Therefore if X is 
basic, then the right side of Eq. (19.1.12) yields the identity matrix. 
You see then that if a basic vertical matrix is premultiplied by its general 
inverse, the product is the identity matrix as indicated in 


ҮХ= І (19.1.14) 


It is most important to remember this rule since it is the foundation of 
Scientific prediction studies. 
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19.2 The Regular Inverse of a Matrix 


But notice now that if X is both square and basie, then both its right 
and left orthonormals must be square, so the right side of Eq. (19.1.6) 
must also yield the identity matrix. We therefore have the rule that if X 
is both basic and square, either pre- or postmultiplieation by its general 
inverse yields the identity matrix. For this particular case we call Y' the 
regular inverse of X. The rule is illustrated by 


XY’ = [1 (19.2.1) 


42.629; (19.2.2) 
But if X is both square and basic wi 


€ normally indicate its regular inverse 
by X^! so that Eqs. (19.2.1) and ( 


19.2.2) would be written as 


XX (19.2.3) 

ХХ =I 19.2.4) 

Therefore, for X both basic and square, we write the inverse from Eq. 
(19.1.3) as in 

X^ = Qai P! (19.2.5) 


From Eqs. (19.2.3) and (19.2.4), а square basic matrix is commutative 
with its inverse, as shown in 


ХХ = X-1x = I (19.2.6) 


: : à clans traditionally do not distinguish between 
different kinds of inverses. They refer to the regular inverse of а matrix 


І call singular matrices. In general, they 52У 
that only nonsingular matric - However, this limited use of 
tricts its usefulness in the human sciences. 


19.3 The Inverse of the Product of Square Basic Matrices 


We may now prove an im 
square basic matrices. We 
matrices a, b, c, and d as in 


portant rule аЬ 


: out the inverse of the product of 
begin by consi 


dering the product of four basic 


X = abed (19.3.1) 


Let us now premultiply both sides of Eq. (19.3.1) by the inverse of a, as in 


aX = (a7ta)beg (19.3.2) 
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But by definition the term in parentheses on the right side here is the 
identity matrix, so we write 


aX = bed (19.3.3) 
Next we premultiply this by the inverse of b. This gives 
ba~ X = (b^!b)cd (19.3.4) 
But from this we have 
b^a^?X = cd (19.3.5) 
Then we premultiply both sides of this equation by c^! to get 
cba X =d (19.3.6) 
Finally we premultiply this by the inverse of d to get 
dc b^a X = І (19.3.7) 
Now we postmultiply both sides of this by the inverse of X to get 
deba XX = X! (19.3.8) 
Or, from this, we have 
а-1077а7 = X^ (19.3.9) 
But from Eq. (19.3.1) we get 
X^! = (abcd) (19.3.10) 
Finally from Eqs. (19.3.10) and (19.3.9), we have 
(abcd)! = d^!c^!b^!a^! (19.3.11) 


Now we could have used this same procedure no matter how many factors 
We had in the product X. Therefore, Eq. (19.3.11) shows that the inverse 
of a product of square basic matrices is equal to the product of the inverses 
1n reverse order. 

According to Eq. (19.3.11), it is also clear that the product of a number of 
matrices cannot have a regular inverse unless each of the factors has а 
regular inverse. This must be true because, as is shown on the left of Eq. 
(19.3.11), the inverse of the products involved the regular inverse of each 


9f the individual factors. 


19.4 Inverses of Special Types of Matrices 


As you will see in the next chapter, the calculation of the inverse of a 
matrix is considerably more complicated than the operation of subtraction, 
addition, and even multiplication. There are, however, certain special 
types of matrices whose inverses are very easy to find. 
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Diagonal matrices 


The inverse of a diagonal matrix is one you encountered very early in 
this book. As you recall, the inverse of а diagonal matrix is simply another 
diagonal matrix whose elements are the reciprocals of the diagonal elements 
of the original matrix. 


Orthonormal matrices 


Since the regular inverse of а matrix is simply а matrix such that when it 
is used as a pre- or postmultiplier, the product is the identity matrix, you 


Equation (19.4.1 ): 
orthonormal. How 


(19.1.3) and take t 
then we have 


X = PII (19.4.2) 
Y' = Пр (19.4.3) 
Or simply 
X=p (19.4.4) 
ү'= р, (19.4.5) 


We may then say that the general inv. 
transpose, and if the orthonormal i 
becomes a regular inverse, 


ix is its 
erse of any orthonormal matrix is it 
S square, then the general inverse 


You should also remember that а permutation matrix is a special case Br 
& square orthonormal matrix, and therefore its inverse is equal to its 
transpose, as shown in 
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is the transpose of the left orthonormal. Suppose we return to Eq. (19.1.1), 
Where X is symmetric. Then assuming also that the basic diagonal is an 
identity matrix, the basic structure of Xis 


X - QIQ' (19.4.7) 
Now notice that Eq. (19.1.3) becomes 
Y' = (QI Q^) (19.4.8) 
But this is simply 
Y' = QIQ (19.4.9) 


Which is the same as (19.4.7). Therefore the general inverse of the major 
Product moment of an orthonormal is that major product moment itself. 


Orthogonal matrices 
Now let us consider the general inverse of a general orthogonal matrix. 
We recall that an orthogonal matrix P is defined as 
PP-D (19.4.10) 


Where the right side is a diagonal matrix. But we have seen previously that 
an orthogonal matrix may be expressed as the product of an orthonormal 
and à diagonal matrix, This is indicated by 


Р = QD"? (19.4.11) 


In Eqs, (19.4.11) and (19.4.10), D is the same diagonal matrix. But we 
Шау regard the right side of Eq. (19.4.11) as the basic structure of P where 
the right orthonormal is the identity matrix. Therefore, we may write the 
Beneral inverse of P as in 

Y = рг) (19.4.12) 


From Eq. (19.4.11) we get 
Рр-! = Q (19.4.13) 


Substituting the left side of Eq. (19.4.13) into the right side of Eq. (19.4.12), 
We get 


Y' = DP’ (19.4.14) 


Therefore, according to Eq. (19.4.14), we see that the general inverse of an 
Orthogonal matrix is obtained by premultiplying the transpose of that 
Matrix by the inverse of its minor product moment. If P is square, then 
“ч. (19.4.14) becomes the regular inverse. It should be remembered in 
Passing that for a square orthogonal matrix, the natural order is defined so 
that the minor product moment yields the diagonal matrix. This means 
that the phim уес{ога: are mutually orthogonal. You also recall that for 
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i i i 1 matrix other than 
ix, if the minor product moment is a diagonal i | 
he entity then the major product moment cannot be a diagonal, ev ar 
ems Pis square. It is clear that if we premultiply P by the right side o 
Eq. (19.4.14), we get 

ud D-P'P = DD = I (19.4.15) 
If we postmultiply P by the right side of Eq. (19.4.14), we get 

PDP! = QDue piping! (19.4.10) 


'The right side of Eq. (19.4.16) comes from Eq. (19.4.11). The diagonal 
factors on the right of Eq. (19.4.16) cancel each other, so we have 


PD^P' = QQ’ (19.4.17) 
If P is not square, then postmultiplication of P by its general inverse gives 
the major product moment of an orthonormal, as in Eq. (19.4.17). If it 15 


square, then the orthonormal must be square, and the right side of Eq. 
(19.4.17) becomes an identity matrix. 


19.5 How to Solve for the Inverse of a Triangular Matrix 


Since all trian, 
element is 0, th 
and (19.1 
simply b 


It is clear from Eqs. gon 
eneral inverse of any matr! 


f we know how to solve for et 
inverse of a triangular matrix, we shall be able to solve for the genera 
inverse of any matrix, and the regular inverse of square basic matrices in 
particular, with consider 


ably less labor than is required for finding the 
basic structure. 


The matrix equations 


First we shall consider а general procedure for finding the inverse of à 
triangular matrix. We let id 


: mau be an upper triangular matrix and Dr ? 
matrix of its diagonal elements. We define the matrix Т” therefore as 


T= Drt (19.5.1) 
where / on the right is the matrix 
diagonal terms. "Therefore the di 
define а matrix B as the inverse 


of the elements of 7" exclusive of the 


agonal elements of / are all 0. Next we 
of T", as in 


В = qua (19.5.2) 


THE INVERSE OF A MATRIX 411 


Postmultiplying both sides of Eq. (19.5.1) by the corresponding sides of 
Eq. (19.5.2), we get 


(Dr -U)B- I (19.5.3) 
From this we have 
РтВ +В = І (19.5.4) 
Premultiplying both sides of this by Dz! we have 
B+ рч В = Dy! (19.5.5) 
From this we get 
—D7"B + "рт = В (19.5.6) 
But Eq. (19.5.6) may be expressed as the product of supervectors, as in 
B 
(-DzU|Dz]|-|-B (19.5.7) 
I 
Suppose now we define the elements in the left factor of the left side of Eq. 
(19.5.7) b 
е = Ч =v (19.5.8) 
апа 
Рӯ = Dy (19.5.9) 


Substituting Eqs. (19.5.8) and (19.5.9) in Eq. (19.5.7) gives 


B 
[v | D.] H =B (19.5.10) 
I 


The expanded equation 
In order to see how Eq. (19.5.10) enables us to solve for B or the inverse 
of T". let us write it out in expanded notation for the case of four variables 


as in Eq. (19.5.11). (19.5.11) 


Ba Ba Вз Ви 
0 v» vg valu 0 0 0 Ba Ba Ba Ви r4 s E x 
O vs v4|0 ж 0 0 =l 5 ü B. By 
0 0 0 valO O w 0 з Ви 
00 0 0l0 0 0 ш 0 0 0 Ba 


1 
0 
0 
0 


Now remember that only the v’s and w’s in Eq. (19.5.11) are known. It 
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5 ? i lve for. We shall see, however, that even though the 
EM left side of Eq. (19.5.11), we need not worry a 
the fact that they are unknown. We carry out the multiplication ro "ü 
left of Eq. (19.5.11) somewhat differently than we ordinarily do. e n 
premultiply the first column of the second factor by the last row 0 hse. 
first factor. We see that this product is 0 and enter the value 0 in the * 
row, first column, on the right of Eq. (19.5.11). Next we premultiply ps 
first column of the second factor by the next to last or third row of t Ө 
first factor. Here again, the product is 0, which we enter in the third TOW, 
first column, of both of the B matrices. In the same way we find that pre- 
multiplication of the first column of the second factor by the second row of 
the first factor also gives 0, which we enter in the second row and first 
column of both of the B matrices, However, multiplication of the first 
column of the second factor by the first row of the first factor does not 
give 0, but rather ш, which is the element in the first row and column of 
the product matrix, namely the element By. 


We now go on to the second column of the postfactor and premultiply it 
by the last row of the prefa 


refactor by the sec 


ond column of the postfactor 
з В». But we have 


e 
already solved for Вз, so W 
can now solve for Ву. 


In the same way, we solve for the elements in the third column of the B 
matrix. You should see then that the method of solution outlined in Eq. 
(19.5.11) depends on the fact that solving for the first unknown element 
requires no knowledge of any of the unknowns, Solving for subsequent 
elements of the B matrix requires a knowledge of only those B elements 
that have been previously solved for. 

By the method of solu (19.5.11), we have proved 
S also an upper triangular 


Numerical example 


Let us now take а simple exam 


ple to show how 
a triangular matrix. We let, 7" 


we solve for the inverse of 
be the matrix j 


n Fig. 19.5.1. 
w $a 
T’=10 3 1 
0. 4 э 


Fie. 19.5.1 
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Then because of Eqs. (19.5.8) and (19.5.9), we define the matrix v as in 


_1 Lo 
5 0 шла] [o -$ 5 
= m 1 
v=} 0 -g o[001|-|o 0 –= 
0 0 -3 000] [б о о 

Fig. 19.5.2 


Corresponding to Eq. (19.5.11), we have 


I uU. M 

2: 6 6 

id 
LE т 1 _1 
ues un | Aon 1] [3 € 9 
0--0- 27 i 

1 1 as 1 

б $ -g|9 g O ———|7|* s “@ 
Үш! И, 0 1 
097 0^ 10:05 00 5 

Ont: co 

бл Л 

Fig. 19.5.3 


Figure 19.5.4 shows that the right side of Fig. 19.5.3 is actually the inverse 
of Pig. 19.5.1. 


NOE 
eae =o a odi 
ue 

Е f 
озо 3 -&|7|]? 
І 
оо 2 [|0 9 5 Pu 
Fra. 19.5.4 


19.6 How to Solve for the Inverse of a Basic Square Matrix 


The matrix equations 


We have seen how to solve for the inverse of a triangular matrix. Now 
let us see how to use this procedure to solve for the inverse of a basic 
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atrix. We recall from Chapter 15 that any basic square T 
cree and columns suitably ordered can be expressed as the product of 8 
аира triangular matrix postmultiplied by an upper triangular matrix. 
о 
This rule is indicated by 


Х = Т.Т, (19.6.1) 


We learned in Chapter 16 how to find these triangular factors if a matrix 1А 
square and basie, апа how to find the partial triangular factors if н е 
matrix is not square or basic. Assume now we have found the triangular 
factors in Eq. (19.6.1). Then the inverse on both sides is 


Куе up (19.6.2) 
But you learned in Section 19.3 that th 


product of the inverses in reverse ord 
may be written as 


e inverse of a product is equal to the 
er, so the right side of Eq. (19.6.2) 


Х- = Тат (19.6.3) 
We then find the inverse 


of X simply by finding its triangular factors, 
then finding the inverse of 


these factors, and finally, postmultiplying the 
gular by the inverse of the lower triangular 
matrix. We shall see, however 


as follows: First let us е 
X 
a=|— (19.6.4) 
I 


Now suppose from Eq. (19.6.1.) and (19.6.4) we write Eq. (19.6.5). 


Tf, 
a= —— 


(19.6.5) 
Та Уы 
But this сап be written аз Eq. (19.6.6). 
T, 
a=|— |r, (19.6.6) 
Ti 


with а supermatrix as the left 


matrix element is the lower triangular f. 
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is the inverse of the upper triangular element. We then find the inverse of 
the upper element Tz, which we already know how to do. Finally using 
Eq. (19.6.3) we get the product of the two inverses, which is the inverse of 
the matrix X. 


Numerical example 


з То illustrate the procedure, let us start with the simple matrix a as 
indicated in Fig. 19.6.1. 


mm 
л 
mn 
© 


oor 
m 
e 


Fia. 19.6.1 


Using the method described in Section 16.5, we get the first residual 
matrix as in Fig. 19.6.2. 


o ui 2 2 0. 0 0 
1 5 10 1 0 3 9 
Т 6 1 © T 5 
ms 2S (ie = 
10 5| |1 | eg ER. 
01 0 0 0 1 cg 
00 1 0 0 0 ! 
Fie. 19.6.2 
The second residual matrix is given by Fig. 19.6.3. 
0 0 0 0 0 0 0 
0 3 9 3 00 0 
De NU 1 00б 2 
a = с Амыз [0 1 3] = 
ü d ed ES, 00 5 
0 1 0 1 00 -3 
Fic. 19.6.3 


Although it is not necessary to calculate the last residual matrix, we do so in 
Fig. 19.6.4 for the sake of completeness. 


6 MATRIX SOLUTIONS 
41 


0 0 0 0 (e 0.0 
0 0 0 0 000 
0. 0 2 2 000 
= - P= hi n 1 = 
s ü 0 5 5 00 20 
00 -3 -9 Ө 10 
0. 0 1 1 000 
Fic. 19.6.4 


From the left side of Figs. 19.6.2 
column vectors for the left, lo 
vectors for the right upper tri. 


› 19.6.3, and 19.6.4, we now collect the 
wer partial triangular matrix, and the jus 
angular matrix, This we show in Fig. 19.6.5. 


2 0 0 

1 3 0 
T, 1 1 2 127281 
puse bes (see T,.—|0 1 3 
d L —=2 5 00 1 

0 1 —3 

0 0 1 

Fig. 19.6.5 


You will notice that We were foresighte 
that its left lower triangular factor is t 
matrix in Fig. 19.5.1. We already sol 
in Fig. 19.5.3, so we may take the 6 
inverse of T, simply by taking the tr. 


d enough to select the matrix X 80 
he transpose of the upper triangular 
Ved for the inverse of this transpose 
Tanspose of this solution to get the 
апзроѕе as in Fig. 19.6.6. 


$ 

5 0 о 

1 H 

e ee Д 
T= 6 3 0 
S 
6 62 

Fie. 19.6.6 


Remember that the inverse of the upper triangular matrix is given in the 
lower submatrix in Fig. 19.6.5. Going back to Eq. (19.6.2) and using the 
matrices from Figs. 19.6.5 and 19.6.6, we have the inverse of the matrix 
X given in Fig. 19.6.7. 
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= 1 3 5 
Row Gg om 0 = 5 
1 1 1 5 3 
Ris крш x» p Ws ко; 
1 т] 1 1 1 
ape 76 ^79 3 6 ^6 3 
bea Sep P 


Fig. 19.6.7 


To show that the right side of Fig. 19.6.7 is actually the inverse of X we 
multiply X as given in the upper submatrix of Fig. 19.6.1 by this inverse, 
as in Fig. 19.6.8. 


эф д o -3 3 100 
TEN 
якъ wr шш 010 
reru: 
== = 5 0 1 
di 6 ^8 2 0 
xa = I 


Fia. 19.6.8 


19.7 How to Solve for the Inverse of a Basic Symmetric Matrix 


The matrix equations 

If X happens to be a basic symmetric matrix, then the solution for the 
Inverse is somewhat more simple than it is for a general square basic 
matrix, If X can be considered as a minor product moment of a basic 
matrix with real elements, then a triangular factoring of this matrix can 
result in upper and lower triangular factors that are transposes of one 
another, For example, if X is a product moment matrix, then Eq. (19.6.1) 
can be written as 

m A ec (19.7.1) 

Then the equation corresponding to Eq. (19.6.5) would be given by Eq. 
(19.7.9) 


X pg". 
а= | |= | — (19.7.2) 
І pepe 
But the right side of this can be written as Eq. (19.7.3). 
ҮЙ 
eS | | (19.7.3) 


gai 
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i iangular-factoring procedure on the super- 
bee doe rif PE AD Sg ui moment as the upper element and 
е ке matrix as the lower element, Eq. (19.7.3) shows that the left 
e ge ular factor consists of an upper element that is the lower 
edat Pe and a lower element that is the transpose of the inverse 


of this triangular factor. For the inverse of the matrix X in Eq. (19.7.1), 
we get 


Arie ncm (19.7.4) 


Numerical example 


We shall present a numerical example of this procedure. First, we define 
the supermatrix a by Fig. 19.7.1. 


D, alt 
2 5-135 
X 15 1 
a=]| -|=| ——— 
I l1 0 0 
01 0 
0 20. д 
Fig. 19.7.1 
Now if we carry out a triangular factoring of this supermatrix the result 
is given in Fig. 19.7.2. 
1 0 0 
2 1 0 
logo ЕС X 
a Ecce: 
52 7: 5110 0 1 I 
0 b 3 
бй ^q 
T, [T] 
Un 
Fre. 19.7.2 


The lower submatrix in the 


Prefactor of Fig. 19.7.2 is the inverse of the 
upper triangular factor, as indicated in Fig. 19.7.3. 


1 —2 5 
T'1al|o L 9 
0 0 1 
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Therefore, using the matrix of Fig. 19.7.3 in Eq. (19.7.4), we get Fig. 
19.7.4 as the inverse of X. 


1 —2 5 1 0 0 30 —17 5 
AU! - |) 1 -3 —2 1 0|2| —17 10 —3 
0 0 1 5 —8 1 5 —3 1 

Fig. 19.7.4 


Then if we multiply the matrix X as given in Fig. 19.7.1 by the inverse as 
given in Fig. 19.7.4, we see that the product is actually the identity matrix. 
This we show in 


LUN 6 30 —17 5 i17 9 

2 5 b]|-17 10 —3|2|0 1 0 

D 5» np 5 =3 1 0 0 1 
Fic. 19.7.5 


19.8 How to Solve for the Product of a Matrix by the Inverse of a Basic 
Square Matrix 


The matrix equations 


In many eases, when we wish to solve a problem involving many variables 
ànd many cases, we are not interested in the inverse of a matrix as such, 
but rather in the product of some matrix by the inverse of another matrix. 
When this is true, we need not first get the inverse of the matrix before 
multiplying the other matrix by it. For example, suppose we have the 


matrix equation 
Ха = Y (19.8.1) 


Here we assume that X апа Y are known matrices and that aisa matrix 
to be determined. Now presumably we could premultiply both sides of 
Eq. (19.8.1) by the inverse of X, assuming that X has a regular inverse. 


This would gi 
vould give m. (19.8.2) 


According to Eq. (19.8.2), we would find the inverse of X and then pre- 
multiply Y by it to find the unknown matrix a. However, we shall proceed 
Somewhat differently. First we write Eq. ( 19.8.1) in the form of the product 
of supermatrices, as in Eq. (19.8.3). 


a 
EIF — |= 0 (19.8.3) 
Next we write the identities 
X eT. (19.8.4) 


and 
Ү = T: (19.8.5) 
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i i ix X is expressed as the product of 
tice that in Eq. (19.8.4) the matrix as | 
ы о: factors. We substitute Eqs. (19.8.4) and (19.8.5) in Eq. (19.8.3) 
to get Eq. (19.8.6) 


a 
[ТЕТ | TZ] 2 = 0 (19.8.6) 
m i 


You notice that a triangular factoring of the prefactor on the left of 
(19.8.6) gives Eq. (19.8.7). 


a 
TT. | 2] B =0 (19.8.7) 
-I 


The triangular factoring yields an upper partial triangular matrix, the left 
submatrix Т, of which is the upper triangular factor of X and the right 
submatrix Z is given by 


Z = T7Y (19.8.8) 
as can be shown direct] dad 
factoring of the supermatrix in the prefactor on the left of Eq. (19.8.3) о! 
Eq. (19.8.6 


which gives the matrix Y 


а 
[Т„ | Z] 5] -0 (19.8.9) 
-I 


We now define the upper triangular matrix Ta as in 


T, = Dp +? (19.8.10) 


ght of Eq. (19.8.10) is simply the diagonal matrix 

made up of the diagonal elements of T, and the right term on the right 18 
« except that all of its diagonal elements аге 0. If we substitute 
), we have Eq. (19.8.11). 


E ) 
[Dr +r |z| —| — 9 (19.8.11 
=I 
Then if we premultiply this equation by Рт, we get Eq. (19.8.12). 
а 
U + DzU | Dgz|| — |. (19.8.12) 
-T 
Next we define 
v= – руци (19.8.13) 


w= DZ (19.8.14) 
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If we substitute Eqs. (19.8.13) and (19.8.14) in Eq. (19.8.12), we have 


a 
[Z —|] В -0 (19.8.15) 
=] 


But multiplying out the left side of this equation we get 
a—va—w-0 (19.8.16) 
Or, from this we get 
va+w=a (19.8.17) 
which may be written as Eq. (19.8.18). 


a 
[ve | w)}-|=e (19.8.18) 
I 


Now let us write Eq. (19.8.18) in expanded notation. This we show in 
Eq. (19.8.19), for the case of X, a 4 by 4 matrix, and a, a 4 by 3 matrix. 


The expanded notation 


ап аш аз 
ал а» аз 


O vr vs 0м | шп Wie Wis || ал аз аз ап аг ал 
O 0 ve voy} Wor Wz Weg || а ас аз|_ |а аз аз 
0 0 0 vy] war Ws Wss || — — n ав аз 
0 0 0 Olwga wn ws 1 0 0 Qa аа адз 
ü X 9 
бф 1 


(19.8.19) 


In Eq. (19.8.19) as in Eq. (19.5.11) the solutions for the a's are made 
Possible because the v submatrix has all zeros in the diagonal and below. 
The essential difference between Eq. (19.5.11) and Eq. (19.8.19) is that 
in Eq. (19.5.11) the right submatrix of the prefactor is diagonal whereas in 
Eq. ( 19.8.19) the right submatrix of the prefactor is a general rectangular 
matrix. The known and unknown submatrices in both equations come in 
the same positions. For example, in both Eq. (19.5.11) and Eq. (19.8.19), 
We have on the right side of the equation the unknown matrix, which is the 
Same as the upper submatrix of the postfactor on the left. In both Eq. 
(19.5.11) and Eq. (19.8.19) it does not matter which column of the unknown 
matrix we solve for first. In the case of Eq. (19.5.11), we started arbitrarily 
With the first column vector of the B matrix and then proceeded on to the 
Temaining column vectors. This order of solution, is not important. But it 
is important that whatever column we start with, the solution proceed 
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This supermatrix is then 
artial triangular matrix as 
riangular matrix, we then 
erse of X, according to the 

We shall not here g 
cases when we wish to multi 


ix that is symmetrical. The 
computational proced iplying апу matrix by the inverse of & 
product-moment mat; d in Chapter 20. 


19.9 How to Solve for the General Inverse of a Nonsquare Basic Matrix 


The solution 
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the transpose of the right orthonormal, and its basic diagonal is the square 
of the basic diagonal of the original matrix. Thus we repeat 


X'X = QwQ' (19.9.3) 
Now let us take the inverse of both sides of Eq. (19.9.3), This gives 
(X'X)7! = QAR (19.9.4) 


since we have already learned that the inverse of any basic product moment 
raised to any power — positive, negative, fractional, or integral — has the 
same orthonormals as the original product moment, and the basic diagonal 
is that of the original raised to the power under consideration. Now if we 
Substitute Eqs. (19.9.1) and (19.9.4) in Eq. (19.9.2), we have 


Y’ = QA"*(Q'Q)AP' (19.9.5) 
But from Eq. (19.9.5), we have 
Ү' = Q(AA)P' (19.9.6) 
And from (19.9.6) we have 
Y= QAP! (19.9.7) 


But notice that Eq. (19.9.7) is exactly the same as Eq. (19.1.3) which has 
been defined as the general inverse of the matrix X. We have therefore 


Proved the rule given by Eq. (19.9.2). | | 
It is of interest to note that in Eq. (19.9.2) if X is both square and basic 


the term in brackets can be written as 

С) т (19.9.8) 
Then if we substitute the right of Eq. (19.9.8) in the right of Eq. (19.9.2), 
We have 

Y= XIX 1X5 (19.9.9) 


or simply 
y'= X- (19.9.10) 


19.10 The Major Product Moment of the Left Orthonormal of a Basic 
Matrix 


The major product moment as a function of the matrix 


Some problems in the human sciences require use of the major product 
Moment of the left orthonormal of a basic matrix. This product moment 
can be expressed as a function of the matrix X by 


PP! = Х(Х'Х)-'Х' (19.10.1) 
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'To prove this equation, we first rewrite it as 
PP! = X[(X'X)x'] (19.10.2) 


i i itute for X its value given in 
the right side of Eq. (19.10.2), we substitu і : i 
a (19.1.1). For the brackets we substitute the right side of Eq. (19.9.6). 


This gives 
PP' = PA(Q'Q)a-!P' (19.10.3) 
But this can be rewritten as 


PP! = Р(дд-)р (19.10.4) 
and this can be written as the identity 


PP’ = pp (19.10.5) 
We have therefore proved Eq. (19.10.1). 

The computational equation 

Let us now solve for this major product, moment of the left orthonormal 
without actually solving for either the inverse of the minor product moment 
of X or for Q directly. 


First we define a supermatrix by Eq. (19.10.6). 


Х'Х 
G= Е (19.10.6) 
X 


s : as the major product moment of a triangular 
matrix, as in 
Х'Х = тт (19.10.7) 
Let us also define a new matrix Z as in 
Z-XmqA (19.10.8) 
Then from this we have 
ZT’! =x (19.10.9) 


Next let us substitute Eqs. (19.10.7) and (19.10.9) in Eq. (19.10.6). This 
gives Eq. (19.10.10). 


TU. 
Gis | (19.10.10) 
ZT. 
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But by factoring out Т” on the right, this can be written as Eq. (19.10.11). 


T 
Gp pm (19.10.11) 
2 


Now Eq. (19.10.11) shows that by applying the triangular factoring pro- 
н to Eq. (19.10.6), we get a left partial triangular factor consisting of 
E wo submatrices as follows: The upper submatrix is the lower triangular 
actor and the lower submatrix Z is simply the matrix X postmultiplied by 
a transpose of. the inverse of this triangular factor. Having solved for Z 
Y this triangular factoring, we can now prove that the major product 
brag of Z is equal to the major product moment of the left orthonormal 
X. From Eq. (19.10.8), we get 

ZZ! = (XT'-)(T1X") (19.10.12) 


But this may be written as 
ZZ! = X(TT')?X' (19.10.13) 


And this, because of Eq. (19.10.7), may be written 
ZZ = Х(Х'Х)!Х' (19.10.14) 


is the same as the right side of Eq. 
d to be the major product moment of 
hat by getting the left 
getting the major 
you automatically 


a the right side of Eq. (19.10.14) 
t oe which we have already prove 
pu orthonormal of X. You see, therefore, t 
рп Vs triangular matrix of Eq. (19.10.5) and then 
"s uct moment of the lower submatrix of this factor, 

the major product moment of the left orthonormal of X. 


The orthonormal matrix Z 
d It is also of interest to note that the minor product moment is an identity 
atrix, as of course it must be if the major product moment is a major 
vera moment of an orthonormal. We can show this, however, very 
ieee by taking the minor product moment of both sides of Eq. (19.10.8) 
In 
ZZ = T-U(X'X)T" (19.10.15) 
But if we substitute from Eq. (19.10.7) in Eq. (19.10.15), we get 
Z'z = (TOT) TT") (19.10.16) 


or simply 
22 = 1 (19.10.17) 


1 ~ : 
9.11 How to Solve for the General Inverse if X Is not Basic 


] inverse of а non- 


Let us next find à simple way of solving for the genera 
We already know 


asic matrix without first solving for its basic structure. 
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i ipli i l inverse is equal to the 
of a matrix multiplied by its general i ‘ 
кз а) dide natie and its inverse. This rule is given by the two equa 
pro а 
it (YiX)" m yx (19.11.1) 
TE 19.11.2) 
(XY) = Хү (19.11. 


If, therefore, we can find a matrix Y which satisfies Eqs. (19.11.1) and 
(19.11.2), we have solved for the general inverse of X. 
We let X be defined by the basic factors given in 


X-u (19.11.3) 
The и and v matrices on the right of Eq. (19.11.3) may be obtained by ine 
triangular factoring method of Chapter 16. We shall now prove that the 


general inverse of Eq. (19.11.3) is given by Eq. (19.11.4), no matter how the 
basic factors in Eq. (19.11.3) are arrived at, 


Y’= v(v'v) (wu) iu’ (19.11.4) 
From Eqs. (19.11.3) and (19.11.4), we get 


VAX = 000) uu)" ит) (19.11.5) 
But this can be written as 


Y'X = оъ) (19.11.6) 


Now from Eqs. (19.11.3) and (19.11.4), we also get 


XY! = ш) uuy (19.11.7) 
But Eq. (19.11.7) can be Written as 


i d 
lution for У” as given in Eq. (19.11.4), would He i 
$: First we would find the u and v factors of X, РГ 


sumably by the simple triangular method of factoring. Defining tw? 
matrices 


J= wo) -y (19.11.9) 
апа 


9 = (uu) (19.11.10) 
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we would solve for f and g by the method indicated in Section 19.8 rather 
than actually finding the inverse of the minor product moments of v and u 
respectively. 


19.12 The Regular Inverse of Certain Special Sums of Matrices 


There are certain special types of sums of matrices that you will meet 
very frequently in the analysis of data. The inverse of the matrices is 
frequently required and may be obtained quite simply in many cases. 


The sum of a basic square matrix and a major product 


First let us consider the type of sum indicated in 


X = (А + ш”) (19.12.1) 


We assume X to Бе square and basic, and we also assume that А is square 
and basic, so that it has a regular inverse. However, we assume the 
matrices и and v’ to have а common dimension less than the order of A. 
We also assume that the inverse of A is already known. The question is 
then, ean we find the inverse of X without actually going through the 
addition indicated in Eq. (19.12.1) and taking the inverse of the new sum? 
If the common dimension of u and v’ is much less than the order of A, we 
can find a simpler procedure for getting the inverse of Х. 

To derive this procedure we first postmultiply both sides of Eq. (19.12.1) 
by xi giving 


(A +w)X7=1 (19.12.2) 
From this we get 
АХ 2I-—wX^ (19.12.3) 
We premultiply both sides of Eq. (19.12.3) by 4^! to get 
X3247- A7w'X^ (19.12.4) 
Next we premultiply this by v' to get 
vX- = vA? — ПАХ! (19.12.5) 
From this we get 
vX- + v Aw = vA? (19.12.6) 


Factoring on the left of Eq. (19.12.6), we get 
(I + v' Aup X = vA (19.12.7) 
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P ultiplying this by the inverse of the parenthesis on the left, we get 
rem 
vX = (I + ›'А-\шу-ъ!А—\ (19.12.8) 
Substituting the right side of Eq. (19.12.8) into the right side of Eq. 
(19.12.4), we have 
X" = A7 — A7w(I фуд) уд (19.12.9) 


19.12.9) involves only the inverse of 
entheses, whose order is the common 


of Eq. (19.12.9) would be 


t than to solve for the Inverse of X in Eq. (19.12.1), 
assuming that the inverse of A is readily available. 


The special case of a major product of vectors 


Аз а special example, we mi 
the analysis of socia] Science da; 
the parentheses in Eq. (19.12. 


in 
£ht take the case frequently aspice 
ta in which и and v аге vectors. In this em 
9) reduce to a scalar quantity and we 


= = A^ huy A71 (19 12.10) 
1ш Pee 8с В . 
x Я 1+ 0А 
The last term on the right of this e 
easy to compute. Therefore, Eq. 
the inverse of the matrix X, provi 


Fig. 19.12.1 
100 1 
uA 86] за тщ 
002 2 
A uw y 
100 + 5 
**-|9" 3 |a 
0 02 6 24 
A шу! 


x 
[ 

о оњ 

оо -— 
N 

о о 


Fig. 19.191 
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From Eq. (19.12.10), then, we have 


1 1 
i 0-0 1 
T Ө zm 0 1 
Pam 3 Mer 
005 
4-1 (A'u) w A7!) 
я 1 +v A'u 
3g 4 X qm OW 
AR NL x ? W^; 
3 1 Seal el eee 
Pe RS OLD ES elle es 
1 IDEE dE cid в 
EE ача я 7 “т и 
Fia. 19.12.2 


А sum of two matrices 
‚ Another type of sum involving inverses is also frequently encountered 
în the analysis of data. Consider the sum given by 
X = а — (a+b) Фат! (19.12.11) 
and also 
Y = a7! — baa +b)! (19.12.12) 
It is easy to show that the right sides of Eqs. (19.12.11) and (19.12.12) can 


Oth be written as 
X =(a+6)" (19.12.13) 


Ү = (де бу (19.12.14) 


то Prove Eq. (19.12.13), we simply substitute the right side of Eq. 
(19.12.13) on the left of Eq. (19.12.11) as in 


(a + 0) = a^ — (a+b) ba (19.12.15) 
We premultiply Eq. (19.12.15) by a + b to get 
I- @+ dort — bat (19.12.16) 


But from this we have 
I = I + фа — фа! = (19.12.17) 


In the same way, we can show by substituting Eq. (19.12.14) in Eq. 
(19.12.12) that an identity results. This substitution we make in 


(a + 0) = a^! — Ба-а + 0) (19.12.18) 
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Equations (19.12.15) and (19.12.18) can also be modified by a reversal of 
О А 12.15) 
signs to give respectively 


(a — b)! = a7! + (a — b)7!ba7! (19.12.19) 
ЖЕ (a — b) = a^! + Ба-а — Ь)! (19.12.20) 


The sum of a matrix and the identity matrix 


When equations of the type given by Eqs. (19.12.15), 1191210, 
(19.12.19), and (19.12.20) are found in actual practice, the matrix a usually 


is the identity matrix. For this particular case, the four equations become 
respectively 


M +b) = I — (I + byb (19.12.21) 
(+b = I-A + b) (19.12.22) 
(I — b) = I + (I — by (19.12.23) 
(=o тЫ (19.12.24) 


It is convenient to be able to rec 
(19.12.24) immediately. Ordinar: 
right of these four equations. 


ognize the identities in Eqs. (10,12,21) е 
ily, the forms encountered are those on 


19.13 The Inverse of a Second-Order Matrix 


Р Nee t 
The inverse of a second-order matrix is much easier to compute than tha 
of higher order matrices, 


: а 
Therefore, we shall show how the inverse of 
second-order matrix may be written at once. If we let 


"- bes x (19.13.1) 
ат а» 
then it is easy to prove that the inverse of a is given by Eq. (19.13.2) 


а = [ 2 —аь 1 (19.13.2) 
=a 9n | апа» — anan 


We simply multiply corres 
in Eq. (19.13.3). 


ai = [а | аљ —@аь 1 (19.13.3) 
аа а» || – an 


Qu | апа» — aran 
Multiplying out the right side of Eq. (19.13.3), we have Eq. (19.13.4) 


pes T G24  — аца + азап 
аа! = LOn Um — азал алар + азе 


апа» — 12021 


aa = f 1 (19.13.5) 


s 
ponding sides of Eqs. (19.13.1) and (19.13.2), * 


(19.13-4) 
or 


0 1 
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Therefore the rule for writing down the inverse of а second-order matrix 
can be stated as follows: 

(1) Write down a new matrix whose diagonal elements are the diagonal 
elements of a in reverse order and whose nondiagonal elements are opposite 
in sign to the nondiagonal elements of a. 

(2) Subtract the product of the nondiagonal elements of a from the 


product of its diagonal elements. 

(3) Divide the elements of the matrix in step (1) by the number cal- 
culated in step (2). This is the inverse of a. 
A numerical example follows. If we have 


9.9 
«2| 1 4 


then steps (1), (2), and (3) are 
4 —2 
ex It 1] 
(2) 3x4—-1x(-2)-214 


LN 
m M 
8) qz 
pugnis MO: 
14 и 
SUMMARY 


l. The general inverse and the basic structure: 

a. The inverse b of a is defined as a matrix such that ab = ba = I. 

b. The general inverse of X = PAQ’ is defined as Y' = CaF 

€. The major product of a matrix X and its general inverse Y’ is 
XY’ = PP’. 

d. The minor product of a matrix and its general inverse is 49:9 = QQ'. 

€. If X is basic the minor product of X with its general inverse is the 
identity matrix, that is, Y'X — I. 

2. If a matrix X is square and basic it is said to have a regular inverse or 
merely an inverse X~!. In this case Y'X = XY‘ and we use the nota- 
tion Y' = X- 

3. The inverse of the product of any number of square basic matrices is the 
Product of their inverses in reverse order, for example, (abcd) = 
d^!c-15-1g71, 

4. Inverses of special types of matrices: 

a. Diagonal matrices: 


1 
= 0 

d ie 70 di 
"a-f ... .jthen2D'-|-.-- ses 
1 
Of xu ida = 
0 a: 
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b. It. "T 

c. Fori а sign matrix, 4! = i. ы | 

d. If О is a square orthonormal, Q = Q'. | ORT 
e. If Q is a vertical orthonormal, its general inverse is 

f 


If P is an orthogonal matrix such that P'P = D is diagonal, its 
general inverse is Y’ = DP’, 


л 


. If T' is an upper triangular matrix and 
T’=Dr+t рч =v  D$g-D, 


the inverse B of T' may be solved from the equation 


B 
b| D]|-|=B 
I 


"т" and 
If X is a square basic matrix with triangular factors X = Т.Т, an 


X 
а=|— Шеп 
1 


the triangular factoring of a is 


e 


The inverse of X is then X~ = T MT. 


-1 


. If X isa basic product тоте 


H . jor 
nt matrix it may be expressed as the та) 
product moment of a triang 


ular matrix; thus X = ТТ". If 


the triangular factors are 


а= | — |Т! 
qu 
and X^! = p-f- 


8. The product of a matrix b 


А be 

Y the inverse of a basic Square matrix ш be 

computed as follows: Let a = X ~'Y and the triangular factors of e 
X TM Ze TpY,T, = 


ne h 
t à Dr à,» = р, w = 04. 
solution for a is obtained from 


a 
v | ш] “|=. 
T 


10. 


13, 
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. The general inverse of a vertical basic matrix X is given by 


ye (А ХУА" 
The major product moment of the left orthonormal of a vertical basic 
matrix 
a. If X = PAQ' then PP’ = X[X'X]X*. 


X'X 
b. IfG = | E X'X = ТТ", then the triangular factoring of G is 
x 


T 
G =| — [T' and PP’ = ZZ'. 
Z 

c. If Z is defined as in (b) then Z'Z = I. 


- The general inverse Y' if X is not basic may be computed as follows: 


Let the partial triangular factoring of X be X — uv’. Then 


Y' = о(р')- (wu) w. 


- The regular inverse of special sums of matrices: 


a. If A is square and basic and X = (A + w’) then 
X`! = A` — A`u(I + vAu) wA. 
b. If u and v in (a) are vectors then 
= 4-1 A7hw'A^! 
роле р + Au 
c. If X = a` (а + 0)- ат and У = а! — Вата + 6)" then 
X — Y = (ad D). 
аз» E 


IG fs =| —an an 
i а тр m 
ад аљ | then а айз — A201 


Chapter 20 


Computations Involving Inverses 


20.1 Introduction 


You have seen in the precedin 
matrix and how, by means of m 
various kinds of matrices. 
product of a matrix by the i 
solving for its inverse. 

In this chapter we shall describe how 
be drawn up and how the com 


g chapter what is meant by the inverse of а 
atrix algebra, we can solve for the inverse of 
You have also seen how we can solve for the 
hverse of another matrix without actually first 


the computational worksheets may 
putations may be organized so as to make pon 
hanical as possible. As indicated in CE. 
inverse of a matrix are in general sors 
using the other three operations of addition, 819” 
traction, and multiplication. 

Many computational Schemes are available for these problems. Some are 


more compact than the methods to be described here. Others require con- 
siderably 


These principles of computational procedure are all directed toward 
making the operations as simple and routine as possible so as to regue 
minimum of attention, concentration, and other higher mental proces 
It has been our experience that the computational procedures can be eer 
mastered by high school graduates of normal intelligence. The chief 5 
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required is that of finger dexterity, such as is needed in typing or playing 
the piano. As a matter of fact, this type of skill is much more important 
than abstract intelligence or the intellectual skills and knowledge acquired 
in college. 

It is not necessary to have a knowledge of either matrix or scalar algebra 
in order to master the computational procedures to be described, even 
though such a knowledge would in general provide greater interest and 
motivation. 


20.2 A New Kind of Vector Multiplication 


The most important principle to be mastered in these computations is 
very simple though perhaps unfamiliar. This is the principle of row-by-row 
or column-by-column multiplication. In matrix algebra you have become 
accustomed to the concept of row-by-column multiplication. The concept 
of row-by-row or column-by-column multiplication is one developed purely 
for computational convenience. Up to now you have been finding the scalar 
Product of two vectors, where a row vector is postmultiplied by a column 
Vector, as in Fig. 20.2.1. 


224158 


C» ке оњ М2 
Ш 
e 
E 


Fie. 20.2.1 


Here, of course, the product is the sum of products of corresponding ele- 
ments of the two vectors. However, in actually carrying out the multiplica- 
tion of these elements, you must be careful to pick out corresponding ele- 
ments from each vector. For example, you must be sure that you do 
multiply the third element in the row vector by the fourth element т the 
Column vector. It is not difficult to avoid this kind of mistake if you have 
Only fifth-order vectors or less to work with, although if you Ps ш 
Scalar products to compute you can readily make such errors. The greater 
the order of the vector, however, the easier it is to multiply the wrong pairs 


of elements together. . 
To avoid making errors of this ki 

Tow directly above the prefactor. 
еп have 


nd, we can first write the postfactor as a 
Using the example in Fig. 20.2.1, we 


9 H a 1-6 
8 #1 6 


Fic. 20.2.2 
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We now get the cumulative product of corresponding numbers тае two 
rows and it is almost impossible to get the wrong pair of numbers in a 
product since one is immediately above the other. | 

Another way we can avoid multiplying the wrong numbers together is to 
write the prefactor in Fig. 20.2.1 as a column immediately before the post- 
factor, as in Fig. 20.2.3. 


9 2 

2 4 

4 3 

E 

5 6 
Fig, 20.2.3 


Again, it would be very diffic 


ult to get the wrong numbers in a product 
because the two numbers to b 


e multiplied together are always side by side. 
Purely for computational convenience, therefore, we may write down two 


vectors whose scalar product is required either as one row above another, or 
one column beside another. Then when We speak of multiplying one row 
by another we mean simply that we sum the products of corresponding ele- 
ments of the two rows. In the Same way when we speak of multiplying two 


columns together we mean that we sum the products of corresponding ele- 
ments in the two columns, 


Now one of the guiding principles in designing the worksheets for the 
computations in this chapter is th 
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In illustrating the worksheets for each of these four problems we shall 
assume for the sake of concreteness that the square matrix a is of order four. 
We shall also assume in cases (1) and (3) that the matrix b has four rows and 
three columns. If you know how to make the calculations for matrices of 
this order you will know how to make them for matrices of any order. The 
methods outlined may be used to advantage for any problem in which the 
order of a is three or more and in which b has any number of columns, in- 
cluding the case in which b is simply a column vector. 

If a is only a second-order matrix, it is better to calculate its inverse 
directly as indicated in Section 19.3. The matrix can then be multiplied by 
this inverse. 


20.3 The Product of a Matrix by the Inverse of a Square Matrix 


Ina large proportion of problems involving the inverse of a matrix, the 
inverse as such is not required, but only the product of some other matrix or 
vector by the inverse. In such cases, it is not necessary, as you saw in 

hapter 19, to compute separately the inverse of the matrix and then to 
multiply another matrix by it. To indicate the computational procedure, 
We shall first show the required worksheets in symbolic form. There will be 
three of these worksheets, which we shall call I, II, and III respectively. 


hese are given in Fig. 20.3.1. 
Worksheet I 


a b Cy 
S Si Cz 
v Z С» 82 


Worksheet III 


R Y eG Ci Si 
Е! =J 0 G 
Fie. 20.3.1 


So that the computations will be easier, it is important that each of the 


MATRIX SOLUTIONS 
438 


rksheets be on a separate sheet of paper. This is true even though 
dia aur of variables involved is small, no greater than three. А 
(p ene m of the symbols in each of the worksheets is as follows: 
nc cm whose inverse is involved in the computations. 
b is the matrix to be premultiplied by the inverse of а. 
Cı is а column of row sums of both a and b. 
S; is a row of column sums of a. 
S; is à row of column sums of b. : 
Ст is the total of C, and also of S; and S% combined. ets 
v is an upper triangular matrix whose diagonal elements are all —1; — 
the upper triangular factor of a. 
Zisb premultiplied by the inverse of the lo 
may be defined as the negative of the rectan, 
triangular factor of the Supermatrix [a | b]. 
С, is a check column obtained by methods to be described later. 


S» is a column of row Sums of both v and Z and Should be the same as C2. 
Worksheet 11 


I is simply the identity matrix of the order of q. 
1 is simply the scalar number 1. 


wer triangular factor of a, un 
gular submatrix of the partia 


OW sums of ш” and should be the same as С^. 
Worksheet IIT 


2 Р ts 
R is a column of the reciprocals of the corresponding diagonal elemen 
of Y. 


H H i in 
Y is a matrix closely related to v in worksheet I which we shall explain ! 
more detail Shortly. 


> Y ined 
X is à matrix closely related to Z in worksheet, 7 ‚ also to be explaine 
more fully, 


Сі is a check column to be explained later, me 
Si is a column of TOW sums of Y and xy combined and should be the sa 
as Ci. 


: ; he 
P" is the transpose of ab; that is, the transpose of b multiplied by * 
nd result of our computations. 


Tesul ао b: 
AT he identity matrix whose order is the width 0 
0 is simply a column of zeros, 


Ја 
К column to be explained presently. Its elements sho" 
all be zero. 
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Worksheet I 


Worksheet II 
X 3 


1011 Un 
W22 


Worksheet III 


R i | 1A 2А ЗА 4А | IB 
В, lA. Yg Yu Ёз Уи Xu 
Ry 2A Yoo Үз You | Xn 
Rs 3A Yss Ys X31 
Rs 4A Кш | Xa 


ІВ | Fu Fs Fa Fa |—1 
Fa 0 


3B | Fi Fes Езз Fas 


Fia. 20.3.2 


Notice carefully now that the vertical rulings of worksheet I should have 
exactly the same spacing as those of III. Furthermore, for purposes of 
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vhich I and III are drawn should be of the 
i he sheets of paper on which 
imn d f e the left margins of the two should be the same. Hence, E 
шкы rulings of the two are the same, the right margins will also 
e 
е that the horizontal rulings of worksheet I should be the PE 
e the horizontal rulings of worksheet II. For purposes of alignment, 
м» of paper on which I and II are drawn should be of the same lengt 
and the top margin of the two should be the same. Therefore, if the ate 
zontal rulings of the two are the same, the lower margins will also be t 
same. | | EN 
The basic plan for the rulings of the worksheets is determined by ms 
sheet I. This can be described in terms of its vertical and horizontal rulings. 


The vertical rulings are spaced as follows from left to right: 
(1) a dummy column t 


© correspond to a functional column in worksheet 

IH 

(2) a column of row designations 

(3) a succession of column. 
or the height of the matrix b 

(4) a succession of column: 
If b is a column vector as it o; 

(5) a check column С 


(6) а column of row sums S to compare with the C column 

The horizontal rulings are spaced as follows from top to bottom: 

(1) a row of column designations i d 

(2) a succession of rows equal in number to the height of the matrix b ап 
the order of the matrix а 


(3) а row of column sums for all entries above it der 
(4) a succession of Tows equal in number to the height of b and the or 
of a. This is the same as (2) above, А as 
For worksheet II the vertical rulings are spaced from left to right 
follows: 


(1) a column of row designations the 
(2) a succession of columns equal in number to the order of a and 
height of b 
(3) a check column С 
(4) a column of TOW sums S 
The horizontal rulings of II 


$ equal in number to the order of the stale” j 
that is to be premultiplied by the inverse - 4 
s equal in number to the width of the matrix b. 
ften may be, this would be but a single column. 


F'or worksheet ITI, you recall, the vertical rulings are exactly the same E 
the vertical rulings of I. However, the horizontal rulings are from toP 
bottom as follows: 

(1) а row of column designations 

(2) a succession of rows e 


: fa 
qual in number to the height of b and order 0 
(3) a succession of rows 


equal in number to the width of b 
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Perhaps the most convenient paper to use for most computational work is 
that which has quarter-inch horizontal rulings and half-inch vertical 
rulings. The size of sheet required will depend, of course, upon the order of 
the matrices involved. Usually, however, such paper is available in both 8i 
by 11 inches and 11 by 17 inches. One of these two sizes can usually be 
adapted to handle most computational projects. 

Once the three worksheets have been ruled up according to the specifica- 
tions of the problem, it is well to cross out with diagonal rulings the spaces 
that are not needed. This takes only a few minutes and will help to prevent 
you from making entries in the wrong positions. Figure 20.3.2 shows where 
these diagonal rulings are drawn. 

The next step is to write in the row and column designations in the row 
апа column labeled i for each of the three worksheets. In each of the three 
Worksheets certain entries can be made that do not depend on the elements 
of the matrices a and b. These are all unit elements as follows: 

(1) In the lower left hand section of worksheet I enter —1 in each 
diagonal position. 

(2) In the upper section of worksheet II enter 1 in each diagonal position. 
Also enter 1 in the cell where the С row and С column intersect. 

(3) In the lower right hand section of worksheet III enter — 1 in each 
diagonal position. Also in this worksheet enter zeros in the lower section of 
the C column. 

You are now ready to write in the elements of the a and b matrices in the 
Upper left and right sections respectively of worksheet I. Once these are 
Written in, you are ready to proceed with the computations a™'b = F. The 
Steps consist of two main groups of operations, the forward solution and the 
back solution. 


The forward solution 
Cycle 1 ; 

(1) In I sum all entries in row 1A of I and enter as C; in column C. This 
Summation includes both the a and the b entries. In the same way, calculate 
and make the entries for C; through Са. Р 

(2) In I sum all entries in column 1A above row C and enter as S; in row 
C. In the same way calculate and make entries for Saz through Sas and also 


for S, through S, 
(8) InI "obe the sum of the entries in column С of I and enter as Cz 


'^ the intersection of row C and column C. | : 
(4) In I. calculate the sum of the entries in row C up to but not DE 
column C and enter as Sr in the intersection of row Cand column S. This 

Entr: ide it. 
у Sr should be the same as Cr best ding positions of row 1A 


From I ll of row 1A into the correspon 
= Worksheet IIT. Therefore, the first row of Y's is the same as the first row 


9f ав and the first row of X’s is the same as the first row of b’s. 
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1A through row C into row 1B of worksheet II. 
M ier ia и tasas ihe first row of w's and S,; is the same 
© C шз with paper clips fasten worksheet III to I so that the — 
rulings are in alignment gs во that the upper edge of worksheet I 
ws 1B and 2B of I. 
P алм the reciprocal if M: T row 1A, column 1A of III and 
it in column R, row 1A o as Ri. . 
Е p^ the entry obtained in step (8) into the keyboard of the lxi 
Multiply Yu by Rı to see that it gives 1. Then multiply succeeding va e» 
in row 1A of III by Zi, and without regard to sign, enter the products in the 
corresponding column positions of row 1B in I. Do this for each entry in the 
row up to but not including the entry in the S column. 

(10) Give each entry in row 1B of I the sign opposite to that of the cor- 
responding entry in row 1A of III if Yn is positive, or the same sign if Yu 
is negative. 

(11) Calculate the sum of all entries in row 
in column 5 as Sy. 
Cycle 2 


(12) Next fold and crease worksh 


1B of I except С and enter 
This entry should be the same as Cy. 


eet II vertically between columns 2 and 
3 and fold the right-hand section under so that only columns 1 and 2 are 
showing, with column 2 at the extreme right. 

(13) Lay column 2 of worksheet, II to the left and directly beside ert 
2A of worksheet I. Be sure that the horizontal rulings of I and П are 11 
alignment. 


(14) Multiply column 2A of worksheet I by column 2 of worksheet H am 
enter the result in row 2A of column 2A, worksheet III, which is still clippe 


tol. This entry will be Y». Note that the only entries currently in colum? 
2 of worksheet II are 1 and wn, 


Therefore, we have for the Ys entry Y» = 
а» + Wawi. t 
(15) Slide column 2 of worksheet II one column to the right on workshee 
I so that it lies immediately to the left of column 3A of I. 
(16) Multiply column 3A 


of I by column 2 of II and enter the result i? 
row 2A, column 3A, of ПІ, as entry Yy, 


(17) Continue sliding column 2 
and get the successive entries on 


тозу 2A of III to, but not including ше 
entry Cx in the C column, and enter in the S column as Sq. This shoul 
the same as Ca immediately to its left. 
(19) Copy Ys from Worksheet II 
(20) Remove worksheet III from I, and turning I face down, fold 2? 
crease between columns 2A and 3A. 
ing on L. These will be at the right, with 2A at the extreme right. 
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(21) With worksheet II turned face up, lay column 2A of worksheet I 
immediately to the left of column 3 of II. Be sure the horizontal rulings cf 
the two sheets are in alignment. 

(22) Multiply column 3 of II by column 2A of I and enter the result in 
row 2B column 3 of II. This will be ws. Currently there are only two 
entries in all of column 3 of П. These are 1 and wa. Therefore ws; = аз + 
21010031. 

(23) If n is greater than 3, slide column 2A of I to the right on П and get 
the entries wy and cs in row 2B of II. This entry c» is obtained in exactly 
the same way as the other entries in row 2B of II. It is column C of II 
multiplied by column 2A of I. Inspection of the entries in these two columns 
Shows that cx = Sas + бэби. 

(24) Sum the entries in row 2B of II up to but not including c» and enter 
in the S column as Sx. This should be the same as сә». 

(25) Calculate the reciprocal of Y» in row 2A of III and enter in the R 
column of III as Re. 

(26) Clip worksheet III to I so that the top edge of III lies between rows 
2B and 3B of I. MN 

(27) Lock Rs in the keyboard and multiply it by Y» to see that it gives js 

(28) Multiply each of the remaining entries to but not including the S 
column in row 2A of III, by R and enter in row 2B of I without regard to 
sign. 

(29) Give each entry in row 2B of I the sign opposite to that of the cor- 
responding entry in row 2A of III if Y» is positive, or the same sign if Yn 
18 negative. | 

(30) Calculate the sum of all entries in row 2B of I except C» and enter in 
column S as S». This entry should be the same as Съ. 

Cycle 8 

(31) Next repeat steps (12) thro 
except that you begin with columns 

ave one more entry at the bottom 


Previously because of the addition of и 
(32) Copy Y from worksheet I II into row 3B, column 3 of worksheet II. 


(33) Repeat steps (20) through (24) to get row ЗВ of worksheet II except 
hat now you begin with column 3A of I and 44 of II. | ч 

(34) Repeat steps (25) through (30) to get row 3B of I except that now 

You begin with row 3A of III and clip worksheet III to I so that its top edge 


lies immediately below row 3B of I. 

Cycle 4 and Subsequent Cycles 
(35) Note that each cycle provi 

Section of III, second, a new row in t 


Tow in the lower section of II. i " 
For each new cycle, the folds in I and II are advanced one column to the 


Tight. To get a new A-row in III, always begin with the columns in I and II 


ugh (18) to get row 3A of worksheet III 
ЗА of I and 3 of II. Note that now you 
of the columns of I and II than you had 
row 2B on both I and III. 


des in order, first, а new row in the upper 
he lower section of I, and third, à new 
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i i B-row of II, merely 
ing to this row. To get the first entry in a j 
plis eren entry of the corresponding A-row in III. To get subsequent 
с wiultply the corresponding columns of II by the column of I cor- 
ding to the row. A 
"Note uas for each new cycle the columns of I and II corresponding in 
number to that cycle, and all columns to the right, each have one more ele- 
ment at the bottom because of the B row that has been added in the pre- 
ceding cycle. | и! 
'The total number of cycles will be equal to the order of the a matrix. 
Notice, however, that in the last cycle no calculations are required for the 
last row of II. There is only one entry wy, and this is Y4 from III. No check 
is required since it has already been checked in III. 
The computations carried out so far constitute what is commonly known 
as the forward solution. We now proceed with the back solution. For m 
we require only the lower sections of worksheets I and III. Remember m 7 
up to now we have made no entries in the lower left section of worksheet III. 


These are the F” entries, Which, as you recall, are those we started out to 
solve for. 


The back solution 


(1) From row 4B of worksheet I, c 
row 1B column 4A of III. Also 
column 4А as Р. Continue un 
tion of I has been copied with 
lower left section of III. 
Cycle 1 

(2) Fold and crease worksheet ITI between rows 4A and 1B so that only 
the lower section of III is showing, and place III so that row 1B lies 1m- 
mediately below row 3B of worksheet I. Besure the vertical rulings of I god 
III are in alignment. E 

(3) Multiply row 3B of I by row 1B of IIT and enter the result as Fs 1 
row 1B of III. As you can see, Ёз = Кары — Zy. 

(4) Slide worksheet IIT up one row so that row 1B of III lies immediately 
below row 2B of I. 


(5) Multiply row 2B of I by row 1B of Ш and enter the result in row 1B 
of III as Ёл. As you can See, Fa = Pay, + Rig Za; 

(6) Slide worksheet ITI up one row so that row 1B of III lies immediately 
below row 1B of I. 

(7) Multiply row 1B of I by row 1 
row 1B of III. The first row of the 

(8) Slide worksheet III up one ro 
below row C of I. с 

(9) Multiply row C of I by row IB of III and enter the result as Gi at үн 
extreme right end of row 1B of III. This entry should be 0 or the same # 


ору Za with sign reversed as Fa into 
copy Zi with sign reversed from I into 
til the last row of Z’s in the lower right sec 
Signs reversed as the last column of F’s in the 


in 
B of III and enter the result as Fu ! 
J’ matrix is now completed. Я 
w so that row 1B of III lies immediately 
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the one immediately to its left. This constitutes a check on the first row of 
the F’ matrix. If the A matrix had been of order greater than four, the same 
procedures would have continued until all elements in the first row of F’ had 
been calculated. 


Cycle 2 
(10) Fold and crease worksheet III between rows 1B and 2B so that only 


rows including 2B and lower are showing. Place III so that row 2B lies 


immediately below 3B of worksheet I. 
(11) Multiply row 3B of I by row 2B of III and enter the result as Fs in 


row 2B of III. 
_ (12) Continue as in steps (4) through (9) to fill in and check the remain- 
ing elements of row 2B of III. 


Cycle 8 
(13) Fold and crease worksheet III between rows 2B and 3B so that only 


row 3B is showing. Proceed as in steps (3) through (9). 


Subsequent Cycles 
(14) If there had been more than three rows to F’ or more than three 


columns to b, subsequent cycles would have followed the same routines. 
For each cycle, worksheet III is folded one row lower than for the preceding 
Cycle. Otherwise the operations within one cycle are the same as for any 
other. 
In many problems, the matrix b consists of а single column vector. In 
Such cases, of course, F” has only one row so that there is only one cycle to 
the back solution. 

A numerical example of the method outlined in this section is given in 
Fig. 20.3.3 for the case in which the a matrix is of order four and the b 


Matrix has four rows and three columns. 


Worksheet I 


3A 4A 1B 2B 


1.160 .920 1.880 .960 

660 1.580 1.000 -980 
4.040 2.220 3.060 1.760 
4.000 1.160 1.020 1.040 


9.860 5.880 6.960 4.740 


— 547 — 434 |— 887 — .453 
1.330 — .084 2.232 633 
—1.000 — .293 |— .891 — .369 
—1.000 |— .828 — .342 


Fia. 20.3.3 (Cont. on next page) 
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Worksheet II 


E: 2 3 4 с s 
pas 

lA| 1 0 0 

2A|0 1 0 

3A| 0 0 1 

4A | 0 0 0 

с 1 

1В| 2.120 3.460 1.740 860 | 8.180 8180 

2B 927 1180 1.864 | 3.970 397 

3B 4.658 6.009 | 10.666 10.667 

4B| =1.130 |-1129 —1.130 

Worksheet III 


Fro. 20.3.3. (Cont.) Product of a Matrix by Inverse of a Square Matrix 


20.4 The Inverse of a General Square Matrix 
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Worksheet I 
1B 2B 3B 4B 


н ооо н 
оон о 
Ноно о 


S 
NNO = 
oo 


Xn 
Xn Хз 1 
Ха Xa 


ex 0 
=] 


Fie. 20.4.1 


You will notice that in both worksheets I and III the number of B 
Columns in the right sections is equal to the number of A columns in the left 
Sections. This must be true since the matrix b is now the identity matrix 
Which is of course square. For the same reason, the number of B rows in the 
lower section of III is the same as the number of A rows in the upper section. 
Therefore, all of the matrices in each of the four quadrants of I and II are 
Square and of the same order. Furthermore, the matrix of Z’s in the lower 
Tight quadrant of I and the matrix of X’s in the upper right quadrant of ш 
are each lower triangular, the latter having diagonal elements of unity. 

Once the worksheets have been ruled up for a matrix of given order, it is 
Dossible to write in the entries of unity at once, irrespective of what the 
elements of the a matrix might be, as follows: ; " | 

In worksheet I, you can write in the 1’s in the diagonal positions in the 
upper right section and the 1’s immediately below in the C row. You can 
also write in the diagonal of — 1’s in the lower left quadrant, just as you did 


in the preceding section. 
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Worksheet 1 


5.980 
8.900 
10.820 
9.200 


ae | 


ІВ | 2.120 3.460 


8.180 
2B 1864 3970 3971 
6.009 10.666 10.667 

—1.130 —1.129 


Worksheet ITI 
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Worksheet II, you recall, is identical to II of the preceding section. 

In worksheet ITI you can write in the 1’s in the diagonal of the upper right 
quadrant. As in the preceding section, you can also write in the —1’sin the 
diagonal of the lower right quadrant. 

In getting the forward solution for the F’ matrix, follow exactly the in- 
structions given from (1) to (35) in the preceding section. However, because 
bis an identity matrix, the computations are somewhat simpler. 

The back solution for the entries in F” is also exactly the same as for the 
back solution instructions in steps (1) through (14) of the previous section. 
Here again the computations are somewhat simplified because the matrix 
of Z'sin the lower right quadrant of I consists of all zeros above the diagonal. 

Precisely the same system of checks is used in both this and the previous 
Section. A numerical example -of the solution of the inverse of a fourth- 
order matrix is given in Fig. 20.4.2. 


20.5 The Product of a Matrix by the Inverse of a Triangular Matrix 


Only rarely do we require the product of a matrix by the inverse of a 
triangular matrix except in the process of getting the product of a matrix by 
the inverse of a square matrix. The procedure is much simpler for a 
triangular than for a square matrix. Two worksheets are required. Work- 
Sheet I is identical in vertical and horizontal ruling with that of worksheet 
III in Section 20.2. Worksheet II is the same in vertical and horizontal 
Tuling to the lower section of worksheet I in Section 20.3. These worksheets 
are illustrated in Fig. 20.5.1 for a, a fourth-order matrix and b, a 4 X 3 
matrix, 


Worksheet I 


Fra. 20.5.1 (Cont. on next page) 
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Worksheet I 


2A 0» аз Qz | Co 
3A азз аз | Сз 
4A аӊ | Cy 


1 
2B |Р, R 0 0 1 
3B FP Fa БВ, 0 1 

1 


ІА |an ay аз аң ] 
4B |F Pos Ез В | 


Sas 


014 
—1 ra 024 


—1 va 
—1 


Fia. 20.6.1 


The elements of the trian 
of worksheet I and the 
steps: 

(1) Sum the rows of 
€ of I. 


А s B r section 
gular matrix a are written into the upper су 
computations proceed according to the follo 


E ; i ]umn 
a in I and enter as С, to C; respectively in co 


wor 
(2) Sum the columns of a including the С column and enter as row C 
II. 


(3) Sum the entries in 
Sr. "This should be the 8: 


jon 
(4) Caleulate the reciprocals of the diagonal entries in the upper т 
of I and enter as R, to T in the diagonal positions in the lower section i: 
(5) Multiply each element in row 1A of I by R, and enter in row 1 
without regard to Sign. Be sure that a, = 1. 


= аф 
(6) Give each element in TOW I of II the sign opposite to its correspo 
ing element in row IA of I. 


ter 
(7) Sum the entries in row 1 of IT up to but not including C; and en 
as Sı. This should be the same as C. 


а cor- 
(8) As in step (7), calculate the remaining rows of II by using the 
responding R from I. 


ter 
row C of II up to but not including C7 and еп 
ame as Су. 
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You are now ready to compute the Ё entries below the diagonal, which 
will complete the computation of а!" Note that the inverse will be given 
in transpose form. 

Row 1B 

Row 1B is already given since it consists only of №, and zeros. 
Row 2B 

To compute F,» take the product of row 2B of I by row 1 of II. But this 
is merely Fi = Rov. 
Row 3B 

First calculate F» by taking the product of row 3B of I by row 2 of II. 
Then calculate P; by taking the product of row 3B of I by row 1 of II, 
remembering that now F» is a part of row ЗВ of I. 

Row 4B 

First calculate F3, by taking the product of row 4B of I by row 3 of II. 
Using row 4B of I as a constant multiplier, move up from row 3 of II a row 
at a time to get Ёз, then Fu. 

Succeeding rows of F 

If a is of larger order, the procedure for getting successive elements of 
each row from right to left is the same. You use the row of I as a constant 
multiplier and move up a row at a time on the rows of II to complete the 
Tow of I. For a large matrix, it is well to fold and crease between rows of 
the lower section of I, so that you can lay the row of I directly above (or 
below) the row of II that is being multiplied. 


The check on the rows of the F’ matrix is obtained by multiplying each 
TOW by row C of II. For each row, the product should be 1, as indicated in 
the lower section of column C of I. You should compute this check for 
cach row of F’ before starting to compute the next row. 

A numerical illustration of the procedure is given in Fig. 20.6.2. 


Worksheet I 


i 1 2 3 4 
1A 2.120 .780 1.160 920 
2A 2.200 .660 1.580 
ЗА 4.040 2.220 
4А |, 1.160 
1B 47170 0 0 0 

2B |— 167 45455 0 0 

3B |— 108 — .074 .24752 0 

4B .061 = 477 — — А .86207 


Ес. 20.6.2. (Cont. on next page) 
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Worksheet II 


í 1 2 3 4 C 5 

С 2.12 2.98 5.86 5.88 16.84 | 16.84 
REST — 368 — 547 — 434 | —2349 |—2.349 
2 =1 — .300 — 718 —2.018 —2.018 
3 -1 = 549 | —1.549 |—1.549 
4 =] —1.000 |—1.000 
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20.7 Multiplication of a Matrix by the Inverse of a Symmetric Matrix 


The computational procedures for multiplying a matrix by the iuto 
à symmetric matrix may be simplified somewhat from those involving sed 
inverse of a general square matrix. In the social sciences, when a pes 
Study requires multiplication of a matrix by the inverse of a matrix, a 
inverse is ordinarily that of a symmetric matrix. In fact, it is adt 
vector derived from the criterion attribute or variable in the m 
matrix. If you know how to carry out the computations for a matrix 
àny order, however, you can certainly do so for a vector. ; rer: 

You have three worksheets as in Section 20.3, but they are quite differe 1 
in design. "The three worksheets are given in symbolic form in Fig. 20.4." 


Worksheet 1 


v z |e6]s&| 


Worksheet 111 
Г P T rod | a, | 


Fic. 20.7.1 
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The meanings of the symbols in each of the worksheets are as follows: 
Worksheet I 

a is the symmetric matrix whose inverse is involved in the computations. 

b is the matrix to be premultiplied by the inverse of a. 

C; is а column of row sums of both a and b. 

S; is а row of column sums of a. 

55 is a row of column sums of b. 

Cr is the total of C, and also of S; and S; combined. 

Ё is а column of the reciprocals of the corresponding diagonal elements of 
Y. 

Y is an upper triangular matrix closely related to a lower triangular 
factor of a. 

X is a matrix to be computationally defined. 

С» is a check column to be computationally defined. 

S» is a column of row sums of both X and Y and should be the same as С». 
Worksheet II 

I is simply the identity matrix whose order is the same as a. 

v is an upper triangular matrix closely related to Y in worksheet I. 

Z is а matrix closely related to X in worksheet I. 

С» is a check column. 

S» is a column of row sums of w' and should be the same as C». 
Worksheet ITI Д 

F” is the transpose of a~'b which is е end result of our computations. 

I is the identity matrix whose order is the width of b. 

0 is a column of zeros. 

G is a final check column whose elements are also zeros. 


Next let us assume that the a matrix whose inverse is involved is of 


Order four and the matrix b that is to be premultiplied by the inverse of a 
has four rows and three columns. The three required worksheets are 


Shown in Fig. 20.7.2. 
Worksheet I 


Sa» баз Sa 


Yu Үз Yu 
Yoo Үз Youu 
Ys Ёз 

Ya 


Fra. 20.7.2 (Cont. on next page) 
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Worksheet II 


Fic. 20.7.2 (Cont.) 


! In ruling up the worksheets for this problem, make sure the rulings 276 
in alignment, just as you did for the problems of the previous sections. The 
rulings for worksheet I determine the rulings for II and III. Both m 
vertical and horizontal rulings of worksheet I are exactly the same а Of 
worksheet I in Section 20.3. Therefore, in drawing up worksheet I for this 
section, follow exactly the specifications as given in that section. 

For worksheet II, both the vertical and horizontal rulings are spaced 
exactly as for worksheet I. 

The vertical rulings of worksheet III are exactly the same as for I and п 
The horizontal rulings provide simply for a row of column designation? 


and a succession of rows equal in number to the width of b, which in the 
example is three. 


out 


After the three worksheets have been ruled up, you should cross 
y whic 


with diagonal rulings those spaces on each of the three worksheets in V 
no entries are to be made, as shown i 
the row and column designations in t 
of the three worksheets. 

Next write in the entries in worksheets II and III that do not depend Us 
the elements of the matrices a and b. In I] these will be a diagonal of a in 
in the lower left-hand quadrant. In III they will be a diagonal of — LP 
the right-hand section and a column of 0’s in the next-to-last column- 

You now write in the elements of a and b matrices in the upper left us 


3 ite iD 
n Fig. 20.7.2. Then you will pet 
he row and column labeled т for © 
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right sections respectively of worksheet I. Since a is symmetric, write in 
only the elements in and above the diagonal. Once the a and b elements 
have been entered, you proceed with the computations for а- = Е. 
As in Section 20.3, the steps consist of two main groups of operations, the 
forward solution and the back solution. 


The forward solution 


(1) Sum all entries in row 1А of worksheet I and enter as C; in column C. 
This summation includes both the a and b entries. In the same way, 
calculate and make the entries for C» through C;. But remember that the 
entries to the left of the diagonal of the matrix a are implied. The sums in 
column С are to include these entries. Since a is symmetrical, the entries 
to the left of the diagonal entry are the same as the corresponding entries 
above the diagonal. Therefore, to get the sum of the entries in a row of a, 
including the unrecorded entries, get the sum total of the recorded elements 
in the row and the entries above the diagonal. 

(2) In I sum all entries in column 1A above row С and enteras Sa: in row 
C. In the same way, calculate and make entries for 5, through Sas and also 
for S, through S». In calculating the S; entries, remember that the entries 
below the diagonal of a are implied and must be included in the column 
Sums, Therefore, to get the sum of the entries in a column of a including 
the unrecorded entries, get the sum total of the recorded elements in the 
column and the entries to the right of the diagonal. 

(3) Calculate the sum of the entries in column С of I and enter as Cz in 
the intersection of row С and column C. 

(4) In I calculate the sum of entries in row C up to but not including 
column С and enter as Sz in thé intersection of row C and column S. This 
entry Sr should be the same as Cz immediately to its left. 

Cycle 1 РА 

(5) In І copy all of row ІА into the corresponding positions of row 1B of T: 
Therefore, the first row of Y's is the same as the first row of a’s. the first 
TOW of X's is the same as the first row of b’s, and Cy is the same as Ci. 

(6) Calculate the reciprocal of Үп in row 1B column ІА, of I, and enter as 
R, in column R, row 1B at the extreme right of I. 

(7) Fold and crease worksheet I between lines 1B and 2B so that every- 
thing below line 1B is turned under and line 1B is the last line showing at 
the bottom of worksheet I. i 

(8) With paper clips, fasten worksheet I over H so that the vertical 
rulings are in alignment and so that line 1B of I lies immediately above 1B 
of II. 

(9) Lock the reciprocal Ri calculated in step (6) into the keyboard of 
the machine. Multiply Yn by Ёз, which is in the keyboard, to see that the 
Product is 1. Then multiply succeeding values in row 1B of I by 7, and 
Without regard to sign, enter the products in the corresponding column 
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; i 
itions of row 1B in II. Do this for each entry in the row up to but no 

positior 1 Cub ists 

i ing the entry in the S column. | " 

шлу each entry in row 1B of II the sign opposite to that of t 

ding entry in row 1B of I. р l оа 

wi) Caleulate the sum of all entries in row 1B of II except Ci» ап 

enter in column S as Sy. This entry should be the same as Cis. 

Cycle 2 


ksheets are in alignment. m 
these two columns together and enter the result as Y» in row 2B, colu 


ND 2A of II. 
nonzero entries in column ZA o 


it lies 
I one column to the right on I so ae unen 
olumn 3A of I. Multiply these two colu 
t as Y in row 2B of I. Ў І 
е column 2А of II one column to the ue 
emaining entries in row 2B of worksheet 1. 


2 in row 2B, column 2A, of I and ente! 
he extreme right of I. that 
Worksheet I between lines 2B and 3B so (ing 
everything below line 2B is turned under and line 2B is the last line show! 
at the bottom of Worksheet I. 


ies 
(19) As in step (8), fasten I over II, except that now line 2B of I lie 
immediately above line 2B of II. hine. 
(20) Lock the reciprocal of У, calculated in Step (17) into the mac TEN 
Multiply Yn by Ri, which is in the keyboard, to see that the product i 


k É ar 
Then multiply Succeeding values in row 2B of I by Re, and without "e 
to sign, enter the products in the corresponding column positions of ro 
inIL Do this for each 


^ м ntry 
entry in the row up to, but not including, the € 
in the S column. 


(21) Give each entry in row 2B of II t 
ponding entry in row 2B of I. nter 
(22) Caleulate the sum of all entries in row 2B of II except C»; and € 
in column S as Sy. This entry should be the same as С». 
Cycle 3 


res- 
he sign opposite that of the сог 
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Cycle 4 and subsequent cycles 

(25) Note that each new cycle provides a new row in the lower section of 
I and a new row in the lower section of II. For each new cycle the fold is 
advanced one column to the right on II. To get a new B row for I, always 
begin with the columns of I and II corresponding to this row. Each new B 
row of II is derived from the latest B row of I. 

In the example given, you have only four cycles, since a is a fourth-order 
matrix. The computations for successive cycles would continue as out- 
lined, however, no matter how large the order of a. 


The back solution 


After all the rows in the lower sections of I and II have been calculated, 
you will have finished with the forward solution and will be ready to start 
the back solution. The back solution is carried out as in Section 20.3, but 
Since the design of the worksheets is somewhat different for a symmetric 
Matrix, the instructions are repeated with the appropriate modifications. 

(1) From row 4B of worksheet II, copy Za with sign reversed as Fa into 
Tow 1, column 4A of worksheet III. Also copy Za» with sign reversed from II 
into column 4A of III as Fẹ. Continue until the last row of Z's in the lower 
right quadrant of II has been copied with signs reversed as the last column 
of F's in the left section of П. 

Cycle 1 

(2) Fold and crease worksheet III between row 1 and the column 
designation row so that the latter is folded under, and place worksheet III 
So that row 1 lies immediately below row ЗВ of worksheet II. Be sure the 
Vertical rulings of II and III are in alignment. . 

(3) Multiply row ЗВ of II by row 1 of ПІ, and enter the result as Ёз in 
TOW 1 of III. As you can see, Fn = Favs — Za Te | 

(4) Slide worksheet III up one row so that row 1 of III lies immediately 

elow г Д : 

(5) MEI A 2B of II by row ur E кек result as P in 
Tow ] i can see, P» = Fsiv23 «UM — Zor А 

(6) ар Mrs III up one row so that row 1 of III lies immediately 

elow . 

(7) Мы ы 1B of II by row 1 of ш and enter the result as P; in 
Tow 1 of III. The first row of the F’ matrix is now completed. 

(8) Next place row 1 of worksheet III immediately below the C row of 
Work 

WE row C of I by row 1 of III and enter the result as Gi at the 
extreme right entry of row 1 of III. This entry should be 0 pre 
9f decima] error, or the same as the vin DONE to its left. This con- 
Stitut с first row of the F” matrix. | 

If the н es ad som greater than four, the same procedures continue 
Until all elements in the first row of Ё” have been calculated. 
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ki Fold and crease worksheet III between rows 1 апа 2 so that row 2 is 


the first row showing at the top of the sheet. Place III so that row 2 lies 
i iately below row 3B of II. - 
шоу?" row 3B of II by row 2 of ТЇЇ and enter the result as Fan 
2 of III. | ч 
a2) Continue as in steps (4) through (9) to fill in and check the r 
maining elements of row 2 of III. 
Cycle 3 
ө) Fold and crease worksheet III between rows 2 and 3 so that 
row 3 is showing. Proceed as in steps (3) through (9). 
Subsequent cycles © 
(14) If there are more than three rows of F’, or more than three Rn = 
in b, subsequent cycles follow the same routines. For each cycle, os xin 
III is folded one row lower than the preceding cycle. Otherwise the ор 
tions within one cycle are the same as for any other. 


: s А ; n da given in 
A numerical example of the method outlined in this section is gV* 
Fig. 20.7.3. 


only 


Worksheet I 


R Ti 1А 2А 3A 4A 1B 
1А | 2.120 780 1.160 -920 1.880 
2А | .780 2.200 .660 1.580 1.000 
3A | 1.160 .660 4.040 2.220 3.060 
4А | .920 1.580 2.220 1.160 1.020 
€ | 4980 5.220 8.080 5.880 6.960 
47170 1B | 2.120 780 1.160 -920 1.880 
.52274 |2B 1.913 233 1.241 308 
29612 |3B 3.377 1.565 1.994 
— 1.30039 4B b. — 469 — .919 


Worksheet II 


Fic. 20.7.3 (Cont. 


. оп next page) 


4,051 2 
IB |-1 = 368 — .547 — 434 _2459| 550 
2B -1 - 422 — 649 | 161 — 328 — -193| 7593, 759! 
-1 590 —озз — 100| 7912. 2 
-1195 — .416 — -300 
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Worksheet III 


i 1А 2А 3A 4А 1B 2B 3B Check 

1 .976 — .619 -037 1.195 | —1 0 0 0 -003 
2 176 .040 .150 .416 0 =] 0 0 :003 
3 .849 .002 — .033 300 0 a | 0 — .004 


Fig. 20.7.3. (Cont.) Product of a Matrix by the Inverse of a Symmetric Matrix 


In most problems involving the multiplication of a matrix by the inverse 
of another, a is a symmetric matrix and b is a vector. If b is a vector, 
worksheet III has only one row in addition to the row of column des- 
ignations. Rather than draw up a separate worksheet, append this row 
to the bottom of worksheet I. In this case, the back solution consists 
simply of the first cycle in the back solution we have just outlined. 


20.8 The Inverse of a Symmetric Matrix 


The computations for the inverse of a symmetric matrix may be simplified 
Somewhat from those of Section 20.4. We need only two worksheets as 


shown in Fig. 20.8.1. 
Worksheet I 


2A ЗА 4A]1B 2B 3B 
à» аз ац |1 0 0 
аз аз аң |0 1 0 
азз аз |0 0 1 
a4 |0 0 0 
Үз Үз Yu|l 0 0 
Yoo Үз You] Ха 1 0 
Үз Үз | Xn Xz 1 
Ya| Ха Хе Ха 
Worksheet П 
i [1A 24 3A 4А |18 2B 3B 
— i 
1A | 1 0 0 0 1 Fu Fe Fis 
2A |0 1 0 0 Ез Fa Fa 
ЗА |0 0 1 0 Pis Ёз Fa 
zd 0 0 0 1 Fu Fa Fa 
1B | —1 v: vs vu |Zu 
2B —1 оз va | Zn Zz 
3B —1 va |Zn Za Zs 
4B =í | Zs 20 Za 


Fic. 20.8.1 (Cont.) 
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Worksheet 11 


1B 2B 3B 

515 — .146 — .126 

146 070 — .393 

126 — .393 017 

048 771 .602 

472 0 0 

192 — .523 0 

-149 036 — .296 

048 — 771 — 602 1.300 


Fie. 20.8.2. (Cont.) The Inverse of a Symmetric Matrix 


Chapter 21 


The Inverse of a Supermatríx 


21.1 Conditions Required for a Regular Inverse 


The simple form of the supermatrix 


You have seen that just as we can take the transposes and add, subtract, 
and multiply simple matrices, so we can also perform these operations with 
Supermatrices. You will doubtless wonder, then, whether we can also take 
the inverse of a supermatrix. We can, of course, do anything with a super- 
matrix that we can with a simple matrix if we first reduce the matrix to 
simple form. However, as you have seen, we can perform the operations of 
transposition, addition, subtraction, and multiplication with the matric 
elements of supermatrices, provided we observe certain rules. It is also 
true that we can express the inverse of a supermatrix in terms of its matric 
elements, provided certain conditions are satisfied. We shall consider only 
the regular inverse of a supermatrix. 

In referring to the inverse of a supermatrix we mean the regular inverse. 
It should be clear, then, that a supermatrix cannot have an inverse unless 
its Simple form hasan inverse. But we know that only basic square matrices 
have regular inverses; therefore, in discussing the inverses of supermatrices, 
We shall restrict ourselves to supermatrices whose simple forms are square 


and basic, 


Restrictions on the supermatrix 

But we shall also place further restrictions on the supermatrices whose 
inverses we shall express as functions of their matric elements. In the first 
place the supermatrix must also be square, which means that the number of 
Matric rows and columns must be equal. And if the supermatrix is square, 
then it has, of course, a principal diagonal of matric elements. Th the 
Second place, we shall require that the simple form of each of the diagonal 
elements also be a square matrix. Third, we specify that each diagonal 
Matric element be basic. о 

Because of the second condition, you can readily see that the supermatrix 
Whose inverse we shall express must be expressible as a symmetrically 
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partitioned square simple matrix. If the simple matrix were not sym- 
metrically partitioned, then of course the diagonal submatrices would not 
be square. Р 3 
We shall consider in detail only the inverse of second-order supermatrices, 
for if we can express the inverse of a second-order supermatrix in terms of 
its matric elements, it is possible, though quite cumbersome, to express 
the inverse of a higher-order supermatrix in terms of its matric elements. 


21.2 The Inverse of a Second-Order Supermatrix 


We shall begin with the supermatrix given by Eq. (21.2.1). 


An Au 
А = ы 21.2.1) 
Ps FA К 


Remember that both A; and А» are square and basic submatrices, but 


they are not necessarily of the same order. We let the inverse of А be 
represented by 


Aca bs a (21.2.2) 


Q21 Q22 
where o; and ax are the same order as Ay and А» respectively. 


Derivation of the inverse 


Our problem now is to express the matric elements in the right side of 
Eq. (21.2.2) in terms of the matric elements in the right side of Eq. (21.2.1). 
Because of Eqs. (21.2.1) and (21.2.2), we have by definition Eq. (21.2-3): 

ha a [s e|. 1, 0 (21.2.3) 
An Аъ | [ол Q22 0 1 
where 7 тапа J, are identity matrices of the order of An and А» respectively: 
Expanding the left side of Eq. (21.2.3), we get the following equations: 


Ann dins m I (21.2.4) 
Anar + Anan = 0 (21.2.5) 
даан Аек = f (212.0) 
Алор + Anan = Г, (21.2.7) 


Ji 
8 
By means of Eqs. (21.2.4) through (21.2.7) we shall now solve for the bur 
in terms of the A's. First we premultiply Eq. (21.2.4) by Au and af 
transposing we get 


Е .8) 
«п = Ag (I — Airan) (212 


Next we premultiply Eq. (21.2.5) by Aj and transpose to get 


9) 
oe = -A Anas (21.2 
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Then we premultiply Eq. (21.2.6) by Az and transposing we get 
an = —An' Anan (21.2.10) 
We also premultiply Eq. (21.2.7) by Az and transpose to get 
a» = А (I — Anan) (21.2.11) 
Now we substitute the right side of Eq. (21.2.8) in the right side of Eq. 
(21.2.10), which gives 
an = —AsAnÁW — Anan) (21.2.12) 
We also substitute the right side of Eq. (21.2.9) in the right side of Eq. 
(21.2.11) and get 
an = Ag + АһА Anan) (21.2.13) 
From Eq. (21.2.12), we get, after a few simple manipulations, 
(An — AnA Anjan = —AnAj! (21.2.14) 
From Eq. (21.2.13) we can also get with a few simple operations 
(An — АзАц А»)ое = I (21.2.15) 
Premultiplying Eq. (21.2.14) by the inverse of the expression in brackets, 
we have 
on = —(An — AnA nAn) Andy (21.2.16) 
We also premultiply Eq. (21.2.15) by the inverse of the term in brackets, 


апа get 
an = (An — АзАц Ан)! (21.2.17) 


If now we substitute the right side of Eq. (21.2.16) in Eq. (21.2.8), we get 


an = Ап + An An(An — AnA Au) Ant! (21.2.18) 
Substituting Eq. (21.2.17) in Eq. (21.2.9) gives 
ою = — Ail An(An — АзАт As)! (21.2.19) 
We now let 
М» = An — AnAj'Ar (21.2.20) 


Then if we substitute Eqs. (21.2.16) through (21.2.20) in Eq. (21.2.2), we 
have 
- -1 = 
2 АТ + АТАМ Аад ~AnAuM; | (21.2.21) 
че -МГАзАп : 

Equation (21.2.21) then gives us the matric elements of the inverse of A in 
terms pe de ese. elements of A. Notice that Eqs. (21.2.16) through 
(21.2.19) assume that M2 given by Eq. (21.2.20) has a regular inverse. 
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Alternate formulas for the inverse 


Next let us consider another solution for the inverse of A, which we shall 
SEO: to be identical with Eq. (21.2.21), but which is expressed somewhat 


differently. We begin by considering a permutation of the matric rows and 
columns of A, as in Eq. (21.2.22). 


a[o Et tM C mam 
É I Jls Assi]. Ts 0 
Multiplying out the right side of Eq. (21.2.22), we have Eq. (21.2.23). 
=|42 An (21.2.23) 
A= |а a, 
We let 
= [2 2s 21.2.24) 
r-[? A ( 


Then Eq. (21.2.22) can be written more compactly as 
A, = тАт' (21.2.25) 


To get the inverse of A, in Eq. (21.2.23), we might have used exactly the 


erse of A in Eq. (21.2.1). Notice that if we 
2 and vice versa in Eq. (21.2.1), we have 
(21.2.23). Therefore, if we use the ww 
f Eq. (21.2.23) as for Eq. (21.2.1), we en 


M,= Ay — AA An (21.2.26) 


Similarly, instead of Eq. (21.2.21), we have 


Az = [* + AnAaM Ава Адам 
—Mi'AnAg Мү! 


| (21.2.27) 
But from Eq. (21.2.25) we have 


(Ay = (Ату (21.2.28) 
Remember that the inverse of a product 
inverses in reverse order, and that the in 
transpose. Therefore the right side of 


of matrices is the product of neni 
verse of a permutation matrix 181 
Eq. (21.2.28) can be written аз 

A; = nA igt (21.2.29) 


But if we premultiply both sides of E 


ipl 
q. (21.2.29) by т’ and postmultiP. М 
by т, we have 


т 


A71 = тА, (21.2.30) 


THE INVERSE OF A SUPERMATRIX 469 


Notice now that the right side of Eq. (21.2.30) means simply that we 
interchange matric rows and columns of Az’. We have therefore, from 
Eqs. (21.2.27) and (21.2.30), Eq. (21.2.31). 


Bie мг MASA 
А! = pv Az! ae Az AnaM A nAz (21.2.31) 


This equation is therefore an alternate form of Eq. (21.2.21) for expressing 
the inverse of a matrix in terms of its matric elements. 


Proof of the identity of both inverses 


We should be able to prove that corresponding elements of Eqs. (21.2.21) 
and (21.2.31) are identical. The identities we shall prove are given by the 


following equations: 


Mj! = Ар + Aj AgM;' Andy (21.2.32) 
МПАьА = АПАМ! (21.2.33) 
AglAnM;! = My'AnAy! (21.2.34) 

Ax! + Az AnaM AnAg = Mz" (21.2.85) 


First we shall prove Eq. (21.2.33). We premultiply both sides of Eq. 
(21.2.33) by M; and postmultiply both sides by М». This gives 


АА М» = MiAi An 
We substitute Eqs. (21.2.20) and (21.2.26) in Eq. (21.2.36) and get 
AnÁg (An — AnAgA:) = (An — ApÁz An) An An (21.2.37) 


(21.2.36) 


Multiplying out both sides of Eq. (21.2.37) gives 


Аш — AnA AnA An = Ap — АА Andy Аз (21.2.38) 


This is, of course, an identity. 
Next we shall prove Eq. (21.2. 
multiply by Mi; to get 


34). We premultiply by М» and post- 


M,Ag An = ААМ (21.2.39) 
We substitute Eqs. (21.2.20) and (21.2.26) in Eq. (21.2.39) and get 

(Ao — AnAA)Ag An = AnAq (An = АА Án) (21.2.40) 
From this we have 


Аз — AnA n Anp Аз = An = АаАпАнАзАз — (21.241) 


Wi t this is an identity. В 
ун Eq. (21.2.32), we first substitute the left side of Eq. (21.2.33) 


into the second term of the right side of Eq. (21.2.32), and get 


My) = Ап + Мг?!АьӊАжАзАт. (21.2.42) 
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Next we pre- and postmultiply this by M; and Ay respectively to get 


An = Mi + AnA An (21.2.43) 
Substituting Eq. (21.2.26) in Eq. (21.2.43) gives 
An = (Au — AnAz An) + AnAz An (21.2.44) 
or 
An = An 


which proves the identity of Eq. (21.2.32). 


Finally we prove Eq. (21.2.35) by first substituting the right side of Eq. 
(21.2.34) into the second term on the left of Eq. (21.2.35). This gives 


Az + Mi'AnAiAnAg = Mz! (21.2.45) 
We pre- and postmultiply Eq. (21.2.45) by Mz and А» respectively to get 
М. + АА! А» = А» (21.2.46) 
Then we substitute Eq. (21.2.20) in (21.2.46), which gives us 
(4n — А44) + AnA An = Ay (21.247) 
or 
Ав = Ав 


Which proves Eq. (21.2.35). 


Applications of the method 


that one or both are diagonal matri 
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Such cases, Eq. (21.2.21) could simplify the operation considerably, and 
might be much simpler computationally than working directly with the 
simple form of the matrix. Such examples can be found in certain experi- 
mental designs in the analysis of variance. 


21.3 Solution When One Diagonal Element Is a Scalar Quantity 


The general solution 

A special case of the inverse of a second-order supermatrix arises when 
one of the diagonal elements is a scalar quantity. This means that Ay and 
Аз are vectors. Let us suppose, for example, that A» is a scalar. Then Ap 
would have to be a column vector and A» a row vector. This we illustrate 


in Fig. 21.3.1. 


Ap 
An 
[x |+ 
Fig. 21.3.1 


Let us now simplify the notation of Eq. (21.2.21) as follows 


Аш => (21.3.1) 
An = uw (21.3.2) 
An = (21.3.3) 
Aid = V (21.3.4) 
АА = U' (21.3.5) 


Substituting Eqs. (21.3.2) through (21.3.4) in Eq. (21.2.20), and drop- 
Ping the subscript 2 on M», we have 


М=а- иу (21.3.6) 


Now M is а scalar quantity, as you can see from the right side of Eq. 
(21.3.6). "Therefore, by means of Eqs. (21.3.1) through (21.3.6), we can 


Write Eq. (21.2.21) as in Eq. (21.3.7). 


KU” «зү 
Alt ye Me 
А1 = р i (21.2.7) 


M M 
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Suppose that we already had the inverse of the matrix Аң. атту. 
we increased the order of An by adding а row anda column, and we w ys e 
to find the inverse of this enlarged matrix. Equation (21.3.7) shows us how 
this can be done readily by using the inverse of An. 


Application for the inversion of simple matrices 


As a matter of fact, therefore, Eq. (21.3.7) suggests a method for getting 
the inverse of any basic square matrix. Suppose we define matrices in terms 
of scalar elements, as in Eq. (21.3.8). 


A= lan] 
Qu an 
ad Be al (21.3.8) 
an ar аз 
Аз = ат аљ az 
Q3 аз da 
ete. 


We also define column and row vectors as in Eqs. (21.3.9) and (21.3.10). 


аз ац 9) 
Uo = а» v = | 1 | % = | а ete. (21.3.5 
аз 
аза 
_ = аз 
us = [as аз] (21.3.10) 


щ = [an аю ав] 


ete. 
Now let us see how we coul » represented by the nth 
equation in Eq. (21.3.8). 


d find the inverse of A 
From Eqs. (21.3.8), (21.3.9), and (21.3.10), we 
have Eq. (21.3.11). 


(21.3.11) 
DM 
из аз 
or we have, in general, Eq. (21.3.12). 
A; = es 0; | (21.3.12) 
U ан 
Suppose now we let 
Жш у; (21.3.13) 
and 
WAS = Ut (21.3.14) 
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Then we would have 


M; = as — ШУ; 


(21.3.15) 


Taking the inverse of both sides of Eq. (21.3.12) and using Eqs. (21.3.7), 
(21.3.13), (21.3.14), and (21.3.15), we have Eq. (21.3.16). 


VU. —Vj 
E —— : 
а А + M; М; (21.3.16 
“= = U; Ey 2 
М; М; 


Equation (21.3.16) shows that we can let 7 take in turn the values 2 to n, and 
so solve for the inverse of any order matrix. We start with Ay’ = 1/an and 
Solve for each higher order А; in sequence according to Eq. (21.2.27). 

This procedure is extremely useful in the behavioral sciences when we 
wish to study the effect of including additional variables with a set of 


variables already selected. 


21.4 The Inverse of a Lower Order Matrix from One of Higher Order 


Deletion of the last row and column 

Suppose now that the inverse of a matrix A, of ordernisgiven. We wish 
to find what the inverse would be of a matrix obtained from An by deleting 
certain rows and an equal number of columns. First we consider the case in 
which the last row and last column of A, has been deleted. We rewrite Eq. 
(21.3.16), using n rather than 7 as a subscript, as Eq. (21.4.1). 


VU, =V, 
Ata T TA 
A = = Ma Ma (21.4.1) 
Б zu d 
M, Ms 
We let 
В = Ах! (21.4.2) 
Now from the right side of Eq. (21.4.1) we have Eq. (21.4.3). 
de Es 
B.n = M. [ 1 | 
(21.4.8) 
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From this we have 


а 


B. B, | 1 eem (21.4.4) 
VB. NB. — NM, ҮМ, 


Or expanding the right side of Eq. (21.4.4), we get Eq. (21.4.5). 


ViU, Wa 
ВВ, _| M. M, (21.4.5) 

Bis zz К, 

M, М, 
Subtracting Eq. (21.4.5) from Eq. (21.4.1), and using Eq. (21.4.2), we get 
p-BsB, [Az 0 (214.6) 

nn 0 0 


ing the last row and column of A ny Eq. jene 
ne rather simply. We divide each clement in the 


matrix obtained by deleting the last row and column of A,. 


Deletion of апу row and column 


But suppose that we are not interested in 
with the last 


e = TÅ ny (21.4.7) 


Now what was the kth row of A is the last r 


ow of C, and what was the ith 


column is the last column. From Eq. (21.4.7) we have 
C; = млг ae 
We let 
Ci =F (21.4.9) 


Now we have, analogous to Eq. (21.4.6), Eq. 21.4.10. 


po E E, “| id (21.4.10) 


Es. NE, 0 о 
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But notice that С„_ is the matrix C with its last row and column deleted. 
Therefore, it is also the matrix A, with its kth row and ith column deleted. 
Then C, on the right of Eq. (21.4.10) is the inverse of the matrix obtained 
by deleting the kth row and ith column of А,. Substituting Eqs. (21.4.2) 
and (21.4.9) in Eq. (21.4.8), we have 

E = т{Вт, (21.4.11) 
We see from this that Z is obtained from B by putting the ith row and kth 


column of B last so that they become the nth row and column. Therefore 
the nth row of E is the same as the ith row of B and the nth column of E is 


the same as the kth column of B. Let us then write 


p – Заві да, (21.4.12) 


Then the right side of Eq. (21.4.12) is the same as the right side of Eq. 
(21.4.10) except for the position of the zeros. The column of zeros in Eq. 
(21.4.12) is in the kth column and the row of zeros is in the ith row. There- 
fore, if we compress the right side of Eq. (21.4.12), we have the inverse of a 
matrix obtained by deleting the kth row and ith column of A n. 


Deletion of any number of rows and columns 

Suppose now we have the inverse of a matrix Am, and we wish to find 
what the inverse would be if we had deleted not only the kth row and ith 
column but also the pth row and gth column. We could calculate first 


pe Вав. = 0 = 4 (21.4.13) 
tk 
and then 
E б.б. doni (21.4.14) 
qp 


In a similar manner, we can calculate from the inverse of a matrix another 
matrix that is the inverse of the original matrix with any number of rows 
and columns deleted. . 

This procedure is useful in the behavioral sciences when we wish to study 
the effect of eliminating variables from a set of variables previously selected. 


SUMMARY 


1. Conditions required for the regular inverse of а supermatrix: 
а. The simple form of the matrix must be square and basic. | 
b. The supermatrix must be à symmetrically partitioned simple matrix. 
e. The diagonal submatrices must be basic. 

2. The inverse of a second order supermatrix: Let 


_ [Аһ a 
mo pr An 
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here An and А» are square and basic. Then the inverse of A is given by 
whe: n à 
the two alternate expressions. 


a Ai + AilAsM; AnA Es d 
| me —M; AnA M; 
where M» = А» — AnA А, and 
b. Мү! = МГА,А | 
dim -Az AnaM А + Ax Mi AnA 
where Mi = An — ААА 


c. The formulas in a and b are most useful when either or both Ay, and 
Аз are diagonal, or when Ах» is a vector. 
3. The case when one diagonal element is a scalar quantity: А 
а. The general solution. Assume Аз a scalar. Let Ар = 0, An = V, 
An = d, A An = V, Andi! = U', M =d — wy. Then 
VU -y 
Ag dT —— — 


-U' Hs 
M M 
b. Application for the inversion of simple matrices. Let 


ij sx а: а; 
А; = SU tase” Knee P 0: = Я u; = [ад Bare а-л], 
ад... а 


1:1): 


“| 
У; = Ail, 


Ui = uA, M; = ay — щҮ: 
Then the formula 
= Va = Vi 
FETA -M M 
у -Ut 1 
M; M 


may be used iteratively for ; — 
simple matrix. 

4. The inverse of a lower ord 

2. Deletion of last row and 

A, by deletion of its 1 


er matrix from one of higher order: 
column. If B = А! 


and Anı is obtained from 
ast row and column, then 


в В.В. _ |28 1 


column from 4 ,. 


Chapter 22 


The General Matríx Reductíon 


Theorems 


22.1 The Reduction Equation 


We shall now consider a theorem that is extremely useful because of its 
many practical applications. It is the basis for most of the methods used in 
factor analysis, a technique applicable in all the sciences whose fundamental 
variables are not generally agreed upon. It serves primarily to aid in dis- 
covering the fundamental variables of a science. 

In the following development, subscripts are used to indicate the order of 
the matrices, the first subscript indicating the number of rows and the 
Second the number of columns. We begin with a general matrix whose rank 


r is defined by 
Ху = XwrXen (22.1.1) 


The arbitrary multipliers 

Next we consider two other matrices Xsy and X,,s. We call X sy the pre- 
multiplier of the general matrix X x, and X ss its postmultiplier. Aside from 
the restrictions of order that we place on these multipliers of the general ma- 
trix, the only restriction we impose is that the triple product Xsy Ху» Xas 
be basic. Then, of course, it must have a regular inverse. This means also 
that S cannot be greater than r. Since these are the only restrictions we 
place on the multipliers of X y, we may also call them the arbitrary pre- and 


postfactors. 


The reduction equation 


We shall next define a new matrix Y y, by 


Уо = Хун (XwnX ns)(XswX wnX ns) '(X swX wn) (22.1.2) 


Study the right side of Eq. (22.1.2) carefully. It looks rather complicated, 
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ting up the equation are simple. You see, of 
n wid = endi ue right nu the general matrix with 
courses : ч rted. The second term on the right, which is to be subtracted 
beu sehr paci consists of a product of three different products. 
e indus in the first parentheses is the general matrix by its postfactor. 
Te pieces in the middle parentheses is the general matrix by its mee n? 
postfactor. Notice, however, that we take the inverse of this triple product. 
The last parentheses include the general matrix by its prefactor. 


22.2 The Orthogonality Theorem and Its Proof 
The first theorem we shall prove is that Y, on the left of Eq. (22.1.2) is 
orthogonal to both the arbitrary multipliers of X Nn, Or that 
ХзуҮх, = 0 (22.2.1) 
Хаа 0) (22.2.2) 
The prefactor 
First we shall prove E 


q. (22.2.1). We premultiply both sides of Eq. 
(22.1.2) by Xsy to get 


XswYws = XsyXy, — Xsw(Xv,X.s)UXswXw,X,s) XsyXw, (22.2) 


But notice that the parentheses with the inverse on the right of Eq. (22.2.3) 
is cancelled by the product in brackets 


on itsleft. "Therefore, we have from 
Eq. (22.2.3) 
XsyYus = XsuXy, — XsNXyn (22.2.4) 
or simply 
XsyYws = 0 (22.2.5) 
Notice that Eq. (22.2.5) is the same as Eq. (22.2.1), which we set out to 
prove. 


The postfactor 


Next we prove Eq. 


т (22.2.2) in a similar manner. This time we post- 
multiply Eq. (22.1.2) by X„s and get 


YysXas = XvaXas — (XoeXns)(XsuXveXns)(XswXwu)Xns) (22.2.6) 
Now the middle parentheses (9 


n the right of Eq. (22.2.6) cancels the product 
following it, and we have 


YuaXas = Xy X, — EX us (22.2.7) 
or simply 


YwnXns = 0 


(22.2.8) 
which is the same as Eq. (22.2.2). 
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22.3 The Minimum Rank of a Product of Two Matrices 


You may recall that in Section 15.7 we stated that the rank of a product 
of two matrices cannot be less than the sum of their ranks less their com- 
mon dimension. At the time we did not prove this theorem. Since it plays 
such an important part in the results of this chapter, we now prove it. 


The minimum rank of the minor product of two basic matrices 
Let us express the rank ¢ of the minor product of two basic matrices by 
Х.Х. = —XatX te (22.3.1) 
where, by definition, the ranks of Xae and Xe are a and c respectively. 


We wish now to find what is the smallest possible value ¢ can have in 
terms of the dimensions of X,, and Xe. We get from Eq. (22.3.1) 


Х.Х. + XaXe = 0 (22.3.2) 
But Eq. (22.3.2) may be written 
A 
e [oes 22.3.3 
к. Xal Ze) =o (233) 
We define 
[Х Хы] = Xa (22.8.4) 
Xel (22.3.5) 
[E] - 
where 
b=et+t (22.3.6) 


Now Xa and X». are basic, since the rank of a supermatrix cannot be less 


i i i ion 15.6). 
than the rank of its submatrix of highest rank (Section 
Substituting the right sides of Eqs. (22.3.4) and (22.3.5) in Eq. (22.3.3) 


gives 
ХьХь = 0 (22.3.7) 
ns, if any, must be placed on the dimensions of 


(22.3.7) to be possible. 
tructures of these two factors by 


Let us see now what restrictio 
Xas and X» in order for Eq. 
First let us define the basic s 
Xab = аЛа 

Xy = QuAQ« 


e matrices are basic, the left orthonormal of Х and the 


(22.3.8) 
(22.3.9) 


Notice that since th 
right orthonormal of X». are both square. 
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Next we substitute the right sides of Eqs. (22.3.8) and (22.3.9) into the 
left of Eq. (22.3.7) and get 
QA QuQuAQ. = 0 (22.3.10) 


We shall first prove that if the minor product of two basic matrices is zero 
then the product of their adjacent orthonormals must also be zero. We pre- 
and postmultiply Eq. (22.3.10) by Q7, and 0, respectively and get 


AsQaQuA, = Q,,0Q.. = 0 (22.3.11) 


Again we pre- and postmultiply Eq. (22.3.11) by Az! 


and Ар! respectively 
and get 


050. = ДОД! = 0 (22.3.12) 


which proves that the product of the adjacent orthonormals is 0. 


Next we consider under what restrictions on the dimensions of the two 
orthonormals, Eq. (22.3.12), can be true. We define a matrix 


б» = [82] (22.3.13) 
Qa 
where 
Qa = Qh (22.3.14) 
Next consider the product 
G = Фан (22.3.15) 
where 
Ока+о) = ©'а+гь (22.3.16) 
From Eq. (22.3.13) and (22.3.15), we get Eq. (22.3.17). 
Qatoda = ba (Qu О] (22.3.17) 
Qe 
where 
Qu = Qh, (22.3.18) 


Multiply out the right side of Eq. (22.3.16) 


Оса аа = dat Jue 22. 3.19) 
950, ©.0ь. ( 
But since the Q в аге orthonor 


mals, and because of Eq. (22.3.12), we can 
write Eq. (22.3.19) simply, 


Оов ао = Es Fal 


20) 
TR E (аза 
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Now the right side of Eq. (22.3.20) is a diagonal matrix, and the left side 
is a product moment matrix. But we know from Section 15.8 that the 
major product moment of a vertieal matrix cannot be diagonal. This means 
that the distinct dimension on the left of Eq. (22.3.20) cannot be greater 
than the common dimension, or 

b>ate (22.3.21) 


Therefore Eq. (22.3.12) cannot hold without Eq. (22.3.21) or the minor 
product of two orthonormals cannot vanish unless their common order is 
greater than or equal to the sum of their distinct orders. 

But the major product of two basic matrices can be 0 only if the product 
of their basic orthonormals is 0. Therefore, the minor product of two basic 
matrices cannot be 0 unless their common order is greater than or equal to 
the sum of their distinct orders. 

From Eqs. (22.3.6) and (22.3.21) 


е+ї>а-+с (22.3.22) 

or 
e+ti-e>at+c-—e (22.3.23) 

or 
i2adc—e (22.3.24) 
But remember that ¢ is the rank of the product given by Eq. D 
wo 


ws that the rank of the minor product o 


Therefore, Eq. (22.3.24) sho à ‘ 
en os a than the sum of their distinct dimensions less 


basic matrices cannot be less 
their common dimension. 

of any two matrices 

product of any two matrices cannot be 
heir common order. Let us define the 


The general case for the product 


We next show that the rank of the 
less than the sum of their ranks less t 


ranks of Xæ and Xv. by 
Xap = Х.Х (22.8.25) 


and 
Хь = XX te (22.3.26) 


be either vertical or horizontal as may also Xə. From 


where now Xa may 


Eqs. (22.3.25) and (22.3.26) we have 
ХъьХь = XoXaXuXc (22.3.27) 


Let 
XoXo = Үч (22.3.28) 
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Then according to our newly proved theorem, the minimum rank of Y,, is 
e 
given by 


Tm >8-++{4—Ь (22.3.29) 
Assuming Y.: of minimum rank, we can then define the rank of Y, by 
Vu Feya (22.3.30) 
Because of Eqs. (22.3.27), (22.3.28), and (22.3.30), 
Aou = X (22.3.31) 
Now let 
Zac YS (22.3.32) 
2. = YX te (22.3.33) 
We show that Z,, in Eq. (22.3.32) is basic by considering a row permutation 
т of Xa, such that 
Xu = La = | I | (22.3.84) 
(a—s)s 


and such that L,, is basic. 
From Eqs. (22.3.32) and (22.3.34) we have 


TX4Y, = ps A = т (22.3.35) 


(a—s)a 


But L,,Y., in the brackets in Eq. (22.3.35) is a limiting case of the minor 
product of two basi i i 


е Zaris also basic. Similarly we show that 
п from Eqs. (22.3.32) and (22.3.33) we re 
write Eq. (22.3.31) as 


DO RM NS (22.3.30) 


where the right of Eq. (22.3.34) defines the minimum rank of the product ОЛ 


theleftasr. And according to Eq. (22.3.29), т cannot be less than the sum 
of the ranks s and t of X and Ха; 


less their common order b. This is wh? 
we set out to prove. 


22.4 The Rank Theorem and Its Proof 


Now we may prove the main the 


is 
orem of this chapter. This theorem ? 
simply that the rank of Y y,, 


e 
defined by Eq. (22.1.2) is exactly equal to 9 
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rank r of Xy, less the smaller order, S, of the arbitrary multiplier. More 


simply, the rank of Y y, is exactly r — S.* 
First, let us substitute the right side of Eq. (22.1.1) for Xy, in Eq. 


(22.1.2). This gives 
Yun = XwrXen — XwrXenXng(XsnXnrXrndns) 'XsNXNXrn (22.4.1) 
On the right side of Eq. (22.4.1) you can now factor X y, on the left and X,n 
on the right. This gives 
Үк. = Ande — XinXns(XswXwrXenXns) XsvXnlXrn (22.4.2) 


To simplify our notation, we let 


X,nXans = Urs (22.4.3) 
XsvXyr = Vsr (22.4.4) 
Substituting Eqs. (22.4.3) and (22.4.4) in Eq. (22.4.2) gives 
Yun = Хм[1„ — U,s(V Ов): sr)X rn (22.4.5) 
Next we let 
Grr = Urs(VssUrs) Vs (22.4.6) 
and 
Wer = In — Grr (22.4.7) 
From (22.4.6) and (22.4.7) we can rewrite (22.4.5) as in 
(22.4.8) 


Yun = XyrWrrXrn 


Let us see now whether we can determine the rank of Ў, exactly. First we 
find the rank of G» in Eq. (22.4.6). Since the parentheses on the right of Eq. 


(22.4.6) has, by definition, а regular inverse, it is basic and its inverse is also 
must be basic, or the parentheses could 


basic and of rank S. Also U+ and Vs 
not have a regular inverse. Now the rank of (Уз, в)! cannot be less 
than S + S — S, or S. Therefore, itis 5. It is easy to show, therefore, from 
the theorems of minimum and maximum ranks of products, that б„ is of 
rank 8. А 

Next let us find the rank t of W,, given by Eq. (22.4.7). We know from 
Section 15.9 that the rank of a sum or difference of two matrices cannot be 
less than the absolute difference between their ranks. Since the rank of I. is 
r and the rank of б is S, we have, therefore, 
{>т—8 (22.4.9) 


Eq. (22.4.9), gives us the minimum rank that W,, can have. 


eterminants is given in Guttman, 
» Psychometrika, Vol. 9 (1944), 


The inequality, 

*A proof of this theorm relyin 
Louis, “General theory and met 
рр. 1—16. 


g heavily on the use of d 
hods of matrix factoring, 
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To find the maximum rank, we first premultiply Eq. (22.4.7) by Vs, to get 


У = Vs — Vs, (22.4.10) 
But from (22.4.6) 
Узб» = УУ) Vs, (22.4.11) 
or 
Egi = s (22.4.12) 


Substituting the left side of Eq. (22.4.12 


) for the second term on the right of 
Eq. (22.4.10), we have 


Vs.W,,=0 (22.4.13) 
Now we know that the rank of a 
greater than the sum of their ra 
of the product in Eq. (22.4.13) i 
rank of W, is t. Since the com 


product of two matrices must be equal to or 
nks less their common order. But the rank 
s obviously 0, the rank of Vs, is S, and the 
mon order of the two factors is r, we have, 


therefore, 
OS sa s (22.4.14) 
or 
S+t-r<0 (22.4.15) 
Subtracting S — r from both sides of Eq. (22.4.15) 
1<т- 5 (22.4.16) 
But the only condition that will satisfy both Eq. (22.4.9) and Eq. (22.4.16) 
is 
t=r=g (22.4.17) 


Therefore, the rank of W,, is exactlyr — S. We indicate the rank of Wrr by 


Wn = WW, (22.4.18) 
and substitute in Eq. (22.4.8) to get 


Yun = Xs WAW, x, (22.4.19) 

We let 
2м = Xy Wn (22.4.20 
Zin = W,X,, (22.4.21) 


We readily prove that Zy: and Zi, are basic by showing that their maximum 
and minimum ranks are 4, Su 


bstituting Eqs. (22.4.20) and (22.4.21) iP 
Eq. (22.4.19) 


Yun = хуй, (22.4.22) 
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But E 

ur (22.4.22) defines the rank of Y x, since the right side is the major 

ae eb of two basie matrices. Therefore, we have proved that the rank 
Nn ls exactly { = r — 5 which we set out to prove. 


22. : 
5 Same Special Cases of the Matrix Reduction Theorem 

, bs. us now consider some interesting special cases of the general matrix 
id uction theorem. You are familiar with one Gi these, although it was not 
identified as such (Chapter 16). 


e; vectors for the arbitrary matrices 


Let us consider, first, а very simpl 


factors. We shall let 
"— I (22.5.1) 


Xas = е; 


5.1) and (22.5.2) in Eq. (22.1.2), we have 


e and special case of the arbitrary 


(22.5.2) 


Substituting Eqs. (22. 


Yun = Жи» ~ Xynej(e.X wes) eX Nn (22.5.3) 
Dropping subscripts on Xyn, we have 
Хе; = Х.; (22.5.4) 
eX = Xi (22.5.5) 
Xe; = Xij (22.5.6) 
Substituting Eqs. (22.5.4), (22.5.5), and (22.5.6) in Eq. (22.5.3) 
ү-х- 52% (22.57) 


You will at once recognize Eq. (22.5.7) as the method outlined in Chapter 
16 for factoring a matrix in order to determineitsrank. You recall that Y in 
Eq. (22.5.7) has zeros in its ith row and jth column. 


Basic structure vectors for the arbitrary matrices 


of the theorem you have already met is closely 
finding the basic structure of a matrix. We let 

Xu, = X'X (22.5.8) 
of a matrix X. But we may write 


Another interesting case 
related to the procedure for 


That is, Xy, is the minor product moment 


X’X in terms of its basic structure, thus, 
X'X = Q5Q' (22.5.9) 
Now we let 
Xsw = Qh (22.5.10) 


Xas = Qu (22.5.11) 
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Substituting Eqs. (22.5.9), (22.5.10), and (22.5.11) in Eq. (22.1.2), we have 
u / n 
Yun = QàQ' — 050'0(0'080/0.)-10',050 (22.5.12) 


But we can readily show that 


QiQ'Q; = êQ (22.5.13) 
REQ’ = &Q', (22.5.14) 
Q':Q5Q'Q., = à (22.5.15) 

Substituting Eqs. (22.5.13), (22.5.14) and (22.5.15) in Eq. (22.5.12) 
Y = QàQ' — àQ.[5]-5,Q', (22.5.16) 

or 

Y = QàQ' — 50.0", (22.5.17) 
or, substituting the left side of Eq. (22.5.9) in the right of Eq. (22.5.17) 
Y = ХХ — 0.0", (22.5.18) 


You recall that Eq. (22.5.18) gives the first residual matrix in the solution 
for the basic structure of X as outlined in Chapter 18. 


Unit vectors for the arbitrary matrices 


A third Simple application of the reduction theorem is closely related to 
о 


€ of the simpler methods of factor analysis known as the centroid method. 
As indicated in previous chapters, factor analysis is of great value to all 
Sciences whose basic variables are not clearly defined. This includes most of 
the social and biological Sciences, as well as Some of the physical sciences. 
S procedures, a matrix of correlation coefficients 15 


usually caleulated from a data matrix and the factor analysis is conducted 
on this matrix. The analysi 


à f 
ngs. To get this first centroid vector by means o 


the reduction theorem, we let Ху, be the matrix of correlation coefficients 
indicated by r, thus, 


Xy, =r (22.5.19) 
The arbitrary factors are simply 
Xsy = 1 (22.5.20) 
Xa-—l (22.5.21) 
Substituting Eqs. (22.5.19), (22.5.20), and (22.5.21) in Eq. (22.1.2) gives 
Y = т n'as (22.5.22) 


or 


ccs fetis l'r 22.5.23) 
ж: (Fa) (=) ; 
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The first centroid vector is, then, 1/1771 or simply a vector made up of row 
(or column) sums of r and divided by the square root of the sum of all the 
elements in r. Equation (22.5.23) gives the first residual matrix, which is 
obtained as you see by subtracting the major product moment of the first 
centroid vector from the correlation matrix. This residual matrix is of rank 
one less than the rank of the correlation matrix, since the smaller dimension 


of the arbitrary factors is in this case simply 1. 
The reduction theorem is the foundation of most factor analysis tech- 


niques, but these are beyond the scope of this book. 


SUMMARY 


1. The reduction equation: Given the matrix Xy, with rank defined by 
Ху, = Ху.Х,,. Given also the arbitrary basic matrices Х„ and Ху 
such that S € rand (XsyXx,X,s) ! exists. A matrix Y y, is defined by 


Yun = Хуп — XnnXns(XswXwnXns)'\XsvXnn 
2. The matrix Yy, is orthogonal to the arbitrary matrices. That is, 
XsyYw, = 0 
YynXns = 0 
3. The minimum rank of the product of two matrices: 


a. The product of two basic matrices cannot be less than the sum of their 


distinct orders less their common order. 
b. The rank of the product of any two matrices cannot be less than the 


sum of their ranks less their common order. 
4. The rank theorem. It is proved that the rank of Y», in paragraph 
1 is precisely S — 7. 
. Special cases of the rank reduction theorem: 
a. If we let Xsy = et and Xys = ез, then 


ET S d 
PER Xi; 


сл 


b. If we let Xy, = X’X = Q8Q', Xsw = Q's, and X,s = Qu, then 
Y = X'X — 0.01 
c. If welet Xy, = r, Xsw = l', and Xas = 1, then 


r =r- (Gra) rn) 


Chapter 33 


Solving Linear Equations 


23.1 Introduction 


Importance for the human sciences 


Most problems in the human sciences are concerned with attempts to 
predict certain criterion variables from certain predictor variables. Typical- 
ly we have measures on a number of predictor variables or attributes for & 
number of entities. We also have measures on one or more criterion vat 
ables for these same entities, We wish to determine the relationships be- 
tween the predictor variables and the criterion variables so that for other 
entities for whom only predictor variables are available we can estimate 
criterion variables. In the human sciences we usually assume that the re- 
lationships between the predictor variables or certain known functions ° 
them and the criterion variables are linear. "Therefore, the problem of pre- 
diction is usually that of finding a vector of weights which, when used as ® 
postmultiplier of the matrix of predictor variables, gives the best estimate 
of a criterion variable. If we have more than one criterion variable, we have 


a vector of prediction weights corresponding to each criterion, or a табх 9 
predictor vectors. 


The fundamental matrix equation 


We may let X y, be a matrix of predictor measures of n variables for each 
of N entities, Zw, be a matrix of criterion measures of m different criteria tor 
each of the N entities, dum be a matrix of vectors of predictor weights for the 
m different criteria. Then the typical problem can be stated in the form 


Хий» — Zym = vm 23.1.) 


where anm is to be solved for, so that the elements of ey, will be as close 1o 
zero as possible. In many problems having to do with the analysis of dat? 
in the human sciences, we also work with secondary or derived matrices 17 
which we have equations of the type given in Eq. (23.1.1), where X and ^ 
are known and a is to be solved for. 
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Therefore, the general problem with which we are concerned in this 
chapter is the solution of a set of linear equations in a specified number of 
unknowns. In matrix notation the general problem consists in solving for a 
in the equation 
Ха— 2 = є (23.1.2) 
where X апа 7 are known and not null. Now according to our practice we 
let X be the vertical order, or in the limiting case, a square matrix. Equa- 
tion (23.1.2) represents the case of the number of scalar equations equal to 
the number of unknowns and also the case of more equations than un- 
knowns. If the number of unknowns is greater than the number of scalar 
equations, instead of Eq. (23.1.2) we may write 

Xa—-Zz-e (23.1.3) 


For both Eq. (23.1.2) and (23.1.3), X may be either basic or nonbasic. 


The four possible conditions 

We shall now consider solutions for a under each of the following four 
conditions of X: (1) It is square and basic; (2) It is vertical and basic; 
(3) It is horizontal and basic; (4) It is nonbasic. Traditionally the mathe- 
maticians have emphasized the fact that an exact solution for a such that e 
is 0 is possible only when X is square and basic. The statisticians have de- 
veloped the ‘‘method of least squares" for the case when X is vertical and 
basic. For the case of more unknowns than equations, the mathematicians 
have been largely content to emphasize that an infinite number of solutions 
exist, while for the case of X nonbasie they have been even less helpful. 


The general inverse solution 
We find that the concept of the general inverse developed in Chapter 19 
can be applied to arrive at a solution for each of the four cases. We recall 


the basic structure of X, 
Ж = PAQ" (23.1.4) 
and its general inverse 
у” = GAP’ (23.1.5) 
We shall then define the optimal solution for a in the case of Eq. (23.1.2) as 
a= YZ (23.1.6) 
and in the case of Eq. (23.1.3) as 
a= YZ (23.1.7) 


Equations (23.1.6) and (23.1.7) apply when X is either basic or nonbasic. 
Substituting Eq. (23.1.5) in Eqs. (23.1.6) and (23.1.7) we have respectively 


a = QAIP'Z (23.1.8) 
a = PAQ'Z (23.1.9) 
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23.2 The Solution When X Is Square and Basic 


The traditional case 


When X is square and basic, and Z is a vector, we have the — 
case in which the number of scalar equations and unknowns 18 —— how 
mathematicians call X here a nonsingular matrix. In no case, р ті 
would we be justified in working with a data matrix that is Men a 
pointed out in Chapter 1, a data matrix, to give scientifically lure re M 
must have many more entities than attributes. However, we here a ў 
examples of derived or secondary matrices in which X is square. ep : * 
as you will recall from Chapter 19 on the inverse of a matrix, we reum ae 
inverses of symmetric basic matrices in order to get the general inverse: Fa 
matrices that are not square, not basic, or neither. Correlation or соу® 


i : ; ly also 
ance matrices are examples of symmetric matrices that are usually 
basic. 


Properties of the solution 


iven by 
It is easy to show that if X is basic and square, the value of a give! 


; 3.1.4) 
Eq. (23.1.6) is an exact solution so that eis 0. We substitute Eqs. (23 
and (23.1.8) in Eq. (23.1.2) and get 


4) 
PAQ'QA"IP'Z — 2 = е er 
or 


PPE—Z-4 (23.2.2) 
But since X is square and basie, P is square, and from Eq. (23.2.2), 
S (23.2.3) 


i , sic is 
The more conventional way to solve for a when X is square and ba 
simply to write 


Xa=Z (23.2) 
then premultiply Eq. (23.2.4) by X^! to get 5) 
XXa = Хд a 
or 
a=X-Z (2328) 
But from (23.1.4) we have 1) 
X^ = [Pag (282. 
ог 
X = Qa P! (2828) 


Substituting Eq. (23.2.8) іп Eq. (23.2.6) gives Eq. (23.1.8) 
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" We have chosen, however, to start with the defined solution of a given by 
Ч- (23.1.8) because of the interesting and useful generalizations we can 
extend to the other types of X matrices. 


23.3 The Solution When X Is Vertical and Basic 


The method of least squares 

When X is vertical and basic, we have the typical case of a data matrix 
with more entities than unknowns. It is for this case that the method of 
least squares was developed to provide a solution for a. This solution 
guarantees that the sum of the squares of the deviations of the predicted 
Z's from the observed Z's is a minimum, or that the trace of e'e in Eq. 
(23.1.2) is a minimum. The method of least squares derived by rather 
complicated methods of the calculus gives as the solution of a 


a = (X'X)?X'Z (23.3.1) 
You have seen, however, in Chapter 19, that the general inverse of X is 


given by 
Y' = (X'Xyx' (23.3.2) 


Properties of the solution 


We now show that the solution for a given by Eq. (23.1.8) does (1) give a 
minimum for the trace of є'є, and also (2) gives an e that is orthogonal to 


X, or 
Xe 0 (23.3.3) 


First we prove Eq. (23.3.3). We substitute Eqs. (23.1.4) and (23.1.8) in 
Eq. (23.1.2) to get 
PAQ’QA"'P'Z — Z = е (23.3.4) 


From (23.3.4) 
(PP' – 1)2 = є (23.3.5) 


Premultiplying both sides of Eq. (23.3.5) by X" 
X'(PP' — 1)Z = X'e (23.3.6) 
Substituting Eq. (23.1.4) in the left side of Eq. (23.3.6) 
QAP'(PP' — 1)Z = X'e (23.3.7) 
or (QAP'PP' — QAP^»Z = X'e 
or (QAP' — QAP')Z = X'e 
or Eq. (23.3.3), X'e = 0, which is what we set out to prove. 


492 MATRIX SOLUTIONS 


Now to show that the trace of e’e is a minimum we need only prove for any 
particular column vector e; that eie; is a minimum. Without loss of 
generality, therefore, we assume that Z and e in Eq. (23.1.4) are vectors. 

First assume that instead of a given by (23.1.8) we take some other vector 
b, where 


b=a+E (23.3.8) 
and write 
Xb-Z=6 (23.3.9) 
We wish to prove now that 
єє > ee (23.3.10) 


for any nonvanishing E in Eq. (23.3.8). F 


irst we get from Eq. (23.3.5) for 
the case of e a vector. 


de = Z'(PP' — 1)(PP’ — 1)z (23.3.1) 
or 


d'e = Z'(I — РР) (23.3.12) 


Next we substitute the right side of Eq. (23.1.4), for X in (23.3.9). We also 


substitute the right side of Eq. (23.1.8) for a in Eq. (23.3.8). Using this 
result for b in Eq. (23.3.9) gives 


PAQ'(QA"'P'Z + E) -Z = (23.3.13) 
From (23.3.13) 


PP'Z + PAQ'E -Z 


= є 
ог 


(PP' — DZ + РАО Е = 6, (23.3.14) 
From Eq. (23.3.14) we get 


66 = [2'(РР' — Г + E'QAP'| [(РР' — DZ+ PAQ'E] (23.3.15) 
or 
ве = Z'(I — PP)Z + 2Z'(PP' — I)PAQ'E 


+ GFQA)AQ'E) (23.3.10 
But 


Z'(PP' — I)PAQ'E — Z'(PAQ' — PAQ’)E = 0 (23.3.17) 
Substituting Eq. (23.3.17) in Eq. (23.3.16) 


вв = Z'U — PP^Z + (E'QAY(AQ'g) (23.3.18) 
Subtracting Eq. (23.3.12) from Eq. (23.3.18) 


єє — e'e = (E'QA)(AQ'E) (23.3.19) 
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But the right of Eq. (23.3.19) is the minor product moment of a vector, 
therefore a sum of squares and positive, hence 


єє > e'e (23.3.10) 


which is the inequality we set out to prove. 


23.4 The Case of the Horizontal Basic Matrix 


An infinite number of solutions 


In the case of Eq. (23.1.3) we have more unknowns than scalar equations 
and if X is basic, there is an infinite number of solutions for a that satisfy 
the equation so that e is 0. This is very easy to prove if we substitute Eqs. 
(23.1.4) and (23.1.9) in Eq. (23.1.3) to get 


QAP'PA-Q'Z — Z = є (23.4.1) 
From Eq. (23.4.1) 
002-2 = є (23.4.2) 
But since X is assumed basic, Q is square and therefore from Eq. (23.4.2) 
e-Z—-Z-0 (23.4.3) 
We could also have substituted Eq. (23.3.2) in Eq. (23.1.7) to get 
a = X(X'X)"Z (23.4.4) 
and then substituted Eq. (23.4.4) in Eq. (23.1.3) to get at once 
Х'Х(Х'Х)12 - 2 = е (23.4.5) 
or 2-2 = е 


An optimal solution 

While it is true that an infinite number of a's exist to satisfy Eq. (23.1.3), 
other than Eq. (23.1.9), this solution does have a very interesting property 
of its own. We can show that in the solution given by Eq. (23.1.9) a'a is a 
minimum. For any other solution, say 


ХЪ= 2 (23.4.6) 
we can prove that 
bb > аа (23.4.7) 
We let 
b= (a4- E) (23.4.8) 


Substituting Eq. (23.4.8) in Eq. (23.4.6), we have 


X'(a + E) = (23.4.9) 


or 
X'a4- ХЕ = 2 (23.4.10) 
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But from Eqs. (23.1.3), (23.4.3) and (23.4.10), we see that. 


X'E-O0 (23.4.11) 
Now from Eq. (23.4.8) 
b'b — a'a 4- 2a'E -- E'E (23.4.12) 
From Eq. (23.1.9) 
аЕ = Z'QAP'E (23.4.13) 
From Eqs. (23.1.4) and (23.4.11) 
QAP’E = 0 (23.4.14) 
Premultiplying Eq. (23.4.14) by A-!Q' gives 
РЕ=0 (23.4.15) 
Substituting Eq. (23.4.15) in Eq. (23.4.13), 
аЕ = 0 (23.4.16) 
Substituting Eq. (23.4.16) in Eq. (23.4.12), 
ЪЪ = аа + E'E (23.4.17) 


But since the second term on the right of Eq. (23.4.17) is a sum of squares, 
we have proved the inequality in Eq. (23.4.6), which we set out to prove. 


23.5. The Solution When X Is Not Basic 


Applications of the nonbasic case 


It is unusual to find a data matrix tha 
matrices whose inverses we ге 
matrices, or else derived from t 
matrices are basic. Usually, 
whose inverses we require ar 
ments of factor analysis, ho 
matrices are not basic. 
squares that assume erro 
criterion variables, we m 
Both because of the prac 
tions for linear equations 


tisnot basic. Typically the derived 
quire are minor product moments of data 
hese. In such cases, the resulting symmetric 
therefore, in the human sciences the matrices 
€ basic. Some of the more advanced develop- 
wever, may involve solutions in which the X 
Also in certain extensions of the theory of least 
TS of measurement in the predictor as well as the 
ау have a derived X matrix that is not basic." 
tical applications and to make the theory of solu- 
more general, it is well to extend the solutions for а 
in Eqs. (23.1.8) and (23.1.9) to the case of X nonbasic. Our development 
will consider in detail only the case of X, а vertical matrix, but the same 
procedures apply to the case of X’, a horizontal matrix. 


*See Horst, Paul, et al., “The Prediction of Ре 


rsonal Adjustment," Social Science 
Research Council, Bull. 48, 1941, pp. 436—444. 
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Properties of the solution 


It is easy to show that the solution for a given by Eq. (23.1.8) gives à 
minimum for both a'a and e'e, and that X is orthogonal to e. We can use ex- 
actly the same proof to show that X is orthogonal to є that we used in Eqs. 
(23.3.3) through (23.3.7), since none of these equations assume that Q is 
square. For the same reason, we can also use Eqs. (23.3.8) through (23.3.19) 
to prove that є'є is a minimum. 

Similarly we can use Eqs. (23.4.8) through (23.4.17) to prove that a'a is a 
minimum, since none of these equations assume that Q is square. 

It should be noted that none of these proofs depend on the width of a and 
Z. Ifaand Z are not vectors, then instead of referring to a'a and єє we talk 
about their traces. 

A method of solving for the general inverse of 
Chapter 19. There you recall that we let 


Х = ш" (23.5.1) 


X, nonbasie, is given in 


rices whose common order defines the rank of X. 


where и and v’ are basic mat: 
eral inverse we have as the 


Then according to the solution given for the gen: 
solution for a 


a = Voor) (uu) wZ (23.5.2) 


Solution if rank of X is known 

d v is given in Chapters 15 and 16, 
if the rank of X is known, is sug- 
the left and right arbitrary multi- 


One method for finding matrices и an 
on the rank of a matrix. An approach, 


gested in Chapter 22. Let us now indicate ; г 
pliers of X as Gz and Gp respectively. We know that if the width of G; and 


the height of Gr are equal to the rank of X, the Y of Chapter 22, Eq. 
(23.1.2) must be 0. Therefore, using this equation, we get 


x= XGa(GrXGr) GX (23.5.3) 

Suppose now we let 
u = XGr(GLXGR)' (23.5.4) 
v = GLX (23.5.5) 


(23.5.4) and (23.5.5) in Eq. (23.5.2), 


пуа XGs(G1XG2) T GLXX G1) GLX 
(23.5.6) 


Then substituting Eqs. 
Y= XGn(GrXGe) (GRX 


which reduces to 


Y'= XG, (Gg, XXO" (6:X Gn) (GX KGa) "G8 (23.5.7) 
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We can show that Eq. (23.5.7) reduces to Eq. (23.1.5) by substituting in 
Eq. (23.5.7) the right side of Eq. (23.1.4). 

Y’ = QAP'GL(GPA'P'G;) (G;PAQ'Gs)(G4QNQ'G.)"GhQAP' (93.5.8) 


Now GP, Q'Gz and their transposes are all Square basie matrices. Suppose 
then we let 


k =GP (23.5.9) 
h = Q'Gr (23.5.10) 
We substitute Eqs. (23.5.9) and (23.5.10) in (23.5.8), and remove parenthe- 


ses to get 
y^z ӨДБ At VEA RR A~2h!-1p', p! 
ог Y' = QAA-?AA-2Ap' 
or Y'= QAP 
which is what we set out to prove. 


Now Gs and GL may be any matrices such that С„ХС» hasa regular in- 
verse. In particular then Suppose the rank of X isr and we partition X into 


x= ES x (23.5.11) 
ar at 


where r + s is the height of X. and r +t its width. Let us now take 


Gi = [Ln | On] (23.5.12) 


1 
Gr = | — (23.5.13) 
On, 


From Eqs. (23.5.11) and (23.5.13) 


ХС c^ [| 


апа 


j (23.5.14) 

From Eqs. (23.5.11) and (23.5.12) 
GLX = [Xn | x,] (23.5.15) 

From Eqs. (23.5.12) and (23.5.14) 
G,XGs = X,, (23.5.16) 


(23.5.17) 


SOLVING LINEAR EQUATIONS 497 


we will have Z,, basic. Then the following will be applicable to Z rather 


than X. 
If now we let 
X» 
Ху = (23.5.18) 
Pes 
and 
Хы = [Х| Хи] (23.5.19) 


then using Eqs. (23.5.16), (23.5.18) and (23.5.19) in Eq. (23.5.7), we have, 
because of (23.5.14) and (23.5.15), 
а = XGí (Xe Xu) Xe(X y Xv) XZ (23.5.20) 


Equation (23.5.20) shows how the general inverse of a matrix of rank r may 
be expressed as a function of the elements in its first r rows and columns pro- 
vided X, is basic. If X, is not basic, it is necessary to choose those r rows 
and columns that do give a basic submatrix 2, in Eq. (23.5.17). 

It should be noted that in some practical problems basic factors of X may 
already be known. In this case Eq. (23.5.2) may be used directly to get the 


general inverse. Sometimes, the basic structure itself is known, in which 


case the general inverse is computed directly from Eq. (23.1.5). 


23.6 The Identity Matrix and the Product of a Matrix by its General 
Inverse 


of a square basic matrix by its inverse is the 
ationship does the product of any matrix by 
y matrix? This problem might be con- 
described in the foregoing sections, in 
Then let us consider the matrix 


(23.6.1) 


By definition, the product 
identity matrix. But what rel 
its general inverse bear to the identit 
sidered a special case of the situation 
which now Z is the identity matrix. 

X = PAQ' 


We indicate the general inverse of X by 
Y' = QAP’ (23.6.2) 


which if Z = Z, is the solution for a in Eq. (23.1.8) in section 23.1. 


If X is vertical and basic 


Assume that X is vertical and basic. Because of Eqs. (23.6.1) and (23.6.2) 


Ү'Х =I (23.6.3) 


We shall now show that in 
xY'-I-E (23.6.4) 
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the trace of E'E is a minimum. This we do by showing for any given 
E that 


EE; is a minimum (23.6.5) 
From Eq. (23.6.4) we have 
XY.—e-E, (23.6.6) 
But from Eq. (23.6.2) 
Y; = QAP; (23.6.7) 
Substituting Eqs. (23.6.1) and (23.6.7) in Eq. (23.6.6) 
PAQ'QA"'P;, — e; = Б, (23.6.8) 
or 
РР; – е = Е, (23.6.9) 
Now from (23.6.9) 
Е.Е; = (PP! — е)(РР, — е) (23.6.10) 
ог 
ЕЕ; = PSP, – 2P, P; 4 1 (23.6.11) 
or 
EE: =1— Pp, (23.6.12) 


Now suppose we had let Y; be some other vector, V; say, where 


Vi= ү, + є; (23.6.13) 
Then substituting Eq. (23.6.7) in Eq. (23.6.13) gives 


Vi = QAP, + с, (23.6.14) 
Substituting Eq. (23.6.1) in Eq. (23.6.6) and using Eq. (23.6.14) instead of 
Eq. (23.6.7), we have 
PAQ'(QAP,, + e) — e; = vE, (23.6.15) 
or 
(PP, — e; 4. PAQ'«) = vE (23.6.16) 


where v ; is the residual ү 
Eq. (23.6.16) 


vEsvEs = GP! — е + CQaP(pp, 
From Eq. (23.6.17) 


ector obtained by using V; instead of Y;.. From 


— e: + PAQ'«) (23.6.17) 


yEvE;—1— РУР + 2P P — e)PAQ'«] + EQAQ'e; (23.6.18) 
But 


(P,P! — &(PAQ'«) = PiAQ'e — P! лое, = 0 (23.6.19) 
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"Therefore Eq. (23.6.18) becomes 


vE'; yE;—1-— PPP. F (eQA) (AQ'e;) (23.6.20) 
From Eqs. (23.6.12) and: (23.6.20) 
vE';vE,; — EE = (eQA)(AQ'e) (23.6.21) 


But the expression on the right of Eq. (23.3.2) is a sum of squares, and 
therefore positive. Thus, any vector other than Y ;. given by Eq. (23.6.7) 
would yield a larger minor product moment than Eq. (23.6.13). This is true 
for all vectors of Y’. Therefore, the trace of E'/E is a minimum when Y” is 


given by Eq. (23.6.2). 


If X is nonbasic 

Next consider the case of X in Eq. (23.6.1) not basic. The demonstration 
in Eq. (23.6.4) through Eq. (23.6.21) would still hold so that the right of 
Eq. (23.6.2) would yield the minimum trace for the minor product moment 
of E in Eq. (23.6.4). But now let us commute the factors in the first term 


on the left of Eq. (23.6.4) to get 


YX-I-F (23.6.22) 


or 
X'Y-I-F (23.6.23) 
X not basic that the trace of the minor product moment of 


We can show for 
23.6.2). This we do by 


F in Eq. (23.6.23) is a minimum for Y' given by Eq. ( | we 
showing for any particular Ё.; that its minor product moment is à minimum. 


From Eq. (23.6.23) we have 


X'Y;—e-F; (23.6.24) 
But from Eq. (23.6.2) 
Ү.; = РАО (23.6.25) 
Substituting Eqs. (23.6.1) and (23.6.25) in Eq. (23.0.24) 
QAP'PA"Q:. — е: = Fi (23.6.26) 
? 00. — e = Ёз (23.6.27) 
From (23.6.27) 
ЕЕ. = (Q.Q' — &)(QQ:. — e2 (23.6.28) 
or 
РР: = 0.0.290: + 1 (23.6.29) 
(23.6.30) 


ог V 
ЕЕ = 1— GQ. 
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i 3.6.1 23.6.21), except that 
ceed exactly as in Eqs. (23.6.13) through C A 
ы Q for P and P for ©, so that corresponding to Eq. (23.6.21), 
et x 
a „ЕУР. — FF; = (&PA)(AP'e,) (23.6.31) 


Equation (22.6.31) shows that any vector other than Y ; given ipla 
(23.6.25) would yield a larger minor product moment than Eq. (23.6.29) 
since the right side of Eq. (23.6.31) is a sum of squares. Р 
Therefore, no matrix exists which when pre- or postmultiplied by X as 
defined in Eq. (23.6.1) gives a better fit to the identity matrix in the least 
square sense than the matrix Y' defined by Eq. (23.6.2). 


In the case of a 
square basic matrix, of course, this fit is perfect. 


SUMMARY 
1. Introduction: 


а. Most scientific problems in the human sciences are concerned with 
predicting criterion variables from predictor v: 


ariables. 
b. If we let 
Ху, be a matrix of known predictor measures of n variables for N 
entities, 


Zw» be a matrix of known criterion measures of m different criteria 
for each of the entities, and 


Anm 


be a matrix of unknown vectors of predictor weights for the m 
different criteria, then the fundamental matrix equation is 
XNndam — Zym = 


ENm 
where anm is to be determined so that, 


tr e'e is a minimum 


c. The X matrix may be one of four different types: (1) square and 
basic, (2) vertical and basic, (3) horizontal and basic, (4) nonbasic. 


d. The general inverse solution: If welet X = PAQ', and У’ = QA`'P’, 
the general solution for a is given by 


() а= Y'ZforXa-z = 


"e 
(2) a = YZ for X'a — Z=0 
2. When X is square and basic: 
a. This is the traditional mathematical case, 
b. The solution for a is a = Х-12, 
с. The solution is exact so that e = 0. 


3. When X is vertical and basic: 
а. This is the usual case for data ma. 


trices in the social sciences. 
b. The general inverse solution for а 


is 
a = (X'X)x'z 


4. 


с 


. When X is nonbasic: 


. The identity matrix and 


SOLVING LINEAR EQUATIONS 501 


c. The solution satisfies the conditions 

(1) X'e-0 

(2) єє = minimum 
The horizontal basic matrix X’: 


a. This case has an infinite number of solutions. 
b. The general inverse solution is 


а= X(X'X)'Z 
c. The solution satisfies the conditions 
(1) e=0 


(2) aa = a minimum 


a. This case is encountered with derived matrices and in certain ad- 


vanced problems in multivariate analysis. 
b. If the rank of X is given by X — wi’ the solution is 
а = v(v'v) (uu) WZ 


c. The solution satisfies the conditions: 
(1) Х'є = 0 
(2) tree = а minimum 
(3) таа = а minimum 
d. If the rank r of X is known and we let 


[Xe Xn es FA Xx sa Xn 
im s Fd sarg) 26 е2 
where r + s and r + t аге the height and width respectively of X, 


then if X» is basic, the general inverse solution of a is 
a= X' cs X1) Xo OG Xv) XN 
uct of а matrix by its general inverse: 


the prod 
PAQ' and its general in- 


Let the basic structure of X be given by X — 


verse by Y' = QAP’. . 
a. Assume X vertical and basic: 


/X — I = E, then E = 0 if Y' is the general inverse of X. 


(1) If Y 
(2) It XY' — I = E then X'E = 0. 
and tr E'E isa minimum if Y’ is the general inverse of X. 
b. Assume X nonbasic: 
СТАЖУ ] = E, then X'E = 0 and tr FE'E is a minimum for ү 
the general inverse of X. 
(2) If x'Y -I= F, ке = 0 and tr F'F is à minimum for Y’ 


the general inverse o 
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Criterion variables, 487, 499 
Cross footing, 154-155 
Cumulative product, 435 


Data matrix, 286-287, 302 
minor product moment of, 288-291, 
302-303 
computational checks, 289-291, 303 
computational steps, 288-289, 302 
row and column sums, 286-287, 302 
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Deletion of rows and columns, 472-474, 
475 
Deviation scores, formula, 271 
sum of, 274 
Diagonal, of a major product, 174-175 
of a matrix, 174 
of a minor product, 176 
principal, 21, 27 
of a product moment matrix, 174 
Diagonal elements, equal, 371-373, 381 
matrie, 69-70, 79 
order of magnitude of, 369-371, 381 
signs of, 368-369, 381 
Diagonal matrices, product of two, 188- 
189, 211 
products involving, 184-188, 210 
sums and differences of, 107, 112 
Diagonal matrix, 33-35, 47, 337, 349 
basic, 368-373, 381 
and trace, 379-380, 382 
and the commutative law, 190, 211 
inverse of, 191-192, 211, 407, 430 
methods of writing, 34, 47 
multiplication by, 184-190 к 
postmultiplication by, see Postmultipli- 
cation, by a diagonal matrix 
powers of, 190-192, 211 
pre- and postmultiplication by, 187, 210 
premultiplication by, see Premultiplica- 
tion, by a diagonal matrix 
single letter designation of, 35, 47 
Square root of, 191, 211 
trace of, 379-380, 382 
unit veetor multiplied by, 187-188 
Diagonal multiplier, 299-300, 304 
computational checks, 299 
computations for, 299 ` 
Diagonal simple matrix, symmetric parti- 
tioning of, 68-69 
Diagonal submatrix, 67 
Diagonal supermatrix, transpose of, 86- 
87, 92 
Difference matrix, 278 
computational checks, 294-295, 303 
computations for, 293-294, 303 
simultaneous computations for, 295, 303 
when N is very large, 297-298, 303-304 
Dimensionalit; › Of matrices orthogonal to 


one another, 327-328, 330 
Dimensions, 20, 27 


common, 147 
distinct, 148, 164 
Distinct orders, 148, 164 
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Distributive law, for algebraic checks, 


159-160, 165 
and matrix multiplication, 157-159, 165 


ei; matrix, 41-42, 48 
multiplication by, 205-207, 213 
postmultiplication by, 205, 213 
pre- and postmultiplication of, 206, 213 
premultiplication by, 205-206, 213 
product of two, 207, 213 

ei vector, minor product moment of, 135- 

136, 138 
multiplieation by, 183-184, 205-207, 
210, 213 
postmultiplication by, 184, 210 
pre- and postmultiplication by, 184, 210 
premultiplication by, 183, 210 

e; vectors, major product of, 136, 139 
minor product of, 135, 138 

Eckhard, Carl, 367 

Elementary vector, 45 

Elements, 21, 27 
in the basic diagonal, equal 


373, 381 
order of magnitude of, 369-371, 381 


signs of, 368-369, 381 
matric, 69, 78, 79, 221, 260 
supradiagonal, 33, 85 
Entities, 5 
administrative, 13 
and attributes, reciprocity of, 8, 15 
experimental, 13 
kinds, 12-13, 15 
rows as, 5-6 
Entry in a matrix, 21, 27 
Equal diagonal elements, 371-373, 381 
Equality, of matrices, 51-52, 59 
of supermatrices, 76-77, 7 
Equation, fundamental matrix, 
499 
Equations, linear, 487-500 
Experimental entities, 13 
External subscript, 75 


lity of, 371- 


487-488, 


Factors, basic, basic matrices аз, 343-344, 


349 
ture of, 367, 380 


basic struc 
ct of, 333-334 


major produ 
of a matrix, 322-323, 329, 33 


order of, 146, 163, 164 

of a product, 333 

of a product momen 
329-330 


3-334, 348 


t matrix, 323-324, 
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scalar, commutativity and, 194, 212 
triangular, 353, 362 
construction of, 354-356, 363 
First residual matrix, 391-392, 400 
Fractional powers, of a product moment 
matrix, 377-378, 381 
Fundamental matrix equation, 487-488, 
499 


General diagonal supermatrix, 69-70, 79 
General inverse, of a basic matrix, 404, 430 
and basic structure, 402-404, 430 
of a general orthogonal matrix, 408- 
409, 431 
major product of a matrix and its, 403- 
404, 430 
minor product of a matrix and its, 404, 
430 
of a nonbasic matrix, 424-426, 432 
of a nonsquare basic matrix, 421-422, 
432 
of an orthonormal matrix, 407-408, 431 
product of a matrix and its, 496-499, 500 
solution for, 488, 499, 500 
of a symmetric nonbasic matrix, 407 
General matrix reduction theorems, 476- 
486 
special cases of, 484-486 
General square matrix, computations for 
inverse, 445-448 
Guttman, Louis, 482 


Height, of a matrix, 20, 27 
of a supermatrix, 63 
Horizontal matrices, 20, 27 
basic, minimum rank for product of, 
343, 349 
Hotelling, Harold, 389 


Idempotent, 274 
Identity matrix, 36, 47 
inverse of, 407, 431 
multiplication by, 197-198, 212 
partitioning of, 69 
powers of, 197-198, 212 
sum of a matrix and, inverse of, 429 
transpose of, 57, 60 
Infradiagonal elements, 33 
as transposes, 85 
Inverse, of a basic produc 
381-382 
of a basic square matrix, 412-416, 431, 


471-472, 475 
product of a matrix by, 418-421, 431 


moment, 378, 
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rse (cont.) 
xe of а major product and, 426- 


421, 432 
of a basic symmetric matrix, 416-418 
computations involving, 433-463 
worksheets, 433 
of a diagonal matrix, 191-192, 211, 407, 
430 
general, see General inverse 
of a general orthogonal matrix, 408- 
409, 431 
of a general square matrix, computa- 
tions for, 445-448 
of the identity matrix, 407, 431 
of a lower order matrix from higher 
order, 472-474, 475 
of a matrix, 402-431 
definition, 402, 430 
general, and the basic structure, 402- 
404, 430 
of orthonormal matrices, 407-408, 431 
of the product of square basic matrices, 
405—406, 430 
regular, 405, 430, 476 
ofa supermatrix, conditions required 
for, 464-465, 474 
of a second-order matrix, 429-430, 432 


of a second-order supermatrix, 465-472, 
474-475 


of a sign matrix, 407, 431 
of special sums of matrices, 426-429, 432 
of special types of matrices, 406-409, 
430-431 
of a square matrix, matrix 
by, 436-445 
of the sum of two matrices, 428-429, 432 
of a supermatrix, 464-475 
of а symmetric matrix, computations 
for, 460-463 
matrix multiplied by, computations 
for, 453-460 
of a triangular matrix, 409-412, 431 
computations for, 450-453 
matrix multiplied by, computations 
for, 448-450 
Inversion of simple matrices, 471-472, 475 
Iterated vectors, scaling constant for, 
389-390 


multiplied 


Left orthonormal, 365, 383-384, 399 
of a basic matrix, major product moment 
of, 422-424, 432 
of a minor product moment, 373 
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solution for, 392-393, 400 
Letter subscripts, 23-25, 28 
Linear equations, 487-500 
general inverse solution for, 488, 499 


Major product, diagonal of, 174-175 
of e; vectors, 136, 139 
of factors, 333-334 
of matrices with same orthonormals, 
375, 381 
of a matrix and its general inverse, 403- 
404, 430 
order of, 134 
of orthonormals, powers of, 320-321, 
329 
trace of, 178, 209 
of two basic matrices, minimum rank of, 
341, 349 
of two vectors, 119-120, 137 
notation for, 119-120 
order characteristics of, 120, 137 
order of, 134 
transpose of, 120-121, 137 
of type I supervectors, 129-131, 137 
transpose of, 131-132, 137 
of type II supervectors, as product of 
two matrices, 144-146, 163 
of type III vectors, 235-237, 262 
notation for, 235, 262 
transpose of, 236-237, 262 
of type IV vectors, 252-256, 263 
notation for, 252-253, 263 
numerical example of, 253-255 
transpose of, 255-256, 263 
Major produet moment, 122-123, 137, 
166, 168, 208 
basic structure of, 373-374, 381 В 
of the left orthonormal of a basic matrix, 
422-424, 432 
of orthonormal matrices, 312-314, 329 
rank of, 346, 350 
trace of, 179, 209 
of type III vectors, 237-240, 262 
column, 237-239, 262 
Tow, 239-240, 262 
of type IV vectors, 256-259, 263-264 
column, 257-259, 263-264 
Tow, 256-257, 263 
of the unit vector, 133-134, 138 
of a vertical matrix, 328, 330 
Major vector product, 360 
rank of, 361-362 
Matric column, 221 
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Matric diagonal elements, in superma- 
trices, 69-70, 79 
Matric elements, 69, 78, 79, 221, 260 
below the diagonal, 68, 78 
Matrices, addition of, 99-112 
equality of, 51-52, 59 
orthogonal to one another, 326-327, 330 
dimensionality of, 327-328, 330 
restrictions on order, 327-328, : 
with same orthonormal 
major product of, 37. 
minor product of, 375-376, 381 
sources of, 14 
subtraction of, 104-105, 112 
Matrix, basie, see Basic matrix 
basic structure of, 364-382 
coordinate of, 21, 27 
correction term, 277 
correlation, see Correlation matrix 
covariance, 33 
from deviation measures, 282, 284 
raw, 277, 283 
data, see Data matrix 
definition, 3, 14 
derived, 4 
designation, methods of, 17, 18-19, 27 
diagonal, see Diagonal matrix 
diagonal of, 174 
difference, see Difference matrix 
dimensions of, 20, 27 
егу, see eij matrix 
entry in, 21, 27 
factors of, 322-323, 329, 333-334, 348 
height, 20, 27 н 
horizontal, see Horizontal matrices 
inverse of, see Inverse, of à matrix 
multiplication of, by unit vector, 182- 
183, 210 - 
nonbasic, see Nonbasic matrix 
nonhorizontal, 365, 381 
nonsingular, 405, 489 
nonsquare basic, general inverse of, 421- 
422, 432 
null, see Null matrix 
order of, 20, 27 ' 
orthogonal, see Orthogonal matrices. 
orthonormal, see Orthonormal matrices 
partial triangular, 39-40, 48, 71-78, 79 
partitioned, 64, 68 r 
permutation, see Permutation matrix 
powers of, 207-208, 213 
prediction data, 12 
rank of, see Rank, of a matrix 
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raw covariance, 277, 283 
rectangular, 30, 46, 56, 60 
regular inverse of, 405, 430, 476 
residual, 360, 362, 363 
first, 391-392, 400 
scalar, see Scalar matrices 
single letter standing for, 25-26, 28 
singular, 405 
size of, 19-21, 27 
square, 20, 30-31, 46 
square basic, see Square basic matrices 
mmetrically partitioned simple, 258 
triangular, see Triangular matrix 
transpose of, 51-81 
as a type II supervector, 142-143, 162- 
163 
vertical, 20, 27 
major product moment of, 328, 330 
width of, 20, 27 
zero, 42, 48 
zero-one, 40-41, 48 
Matrix multiplication, 142-165 
associative law of, 150-153, 164 
commutative law of, 149-150, 164 
distributive law in, 157-159, 165 
see also Multiplication 
Matrix reduction theorem, special cases 
of, 484-486 
Maximum rank, of a product, 338-339, 
349 
of products of two basic matrices, 339- 
340, 349 
of the sum of any number of matrices, 
347, 350 
of the sum of two matrices, 346-347, 350 
Mean, 268-269, 282 
Minimum rank, of the major product of 
two basic matrices, 341, 349 
of the minor product of two basic ma- 
trices, 341-342, 349, 478-480 
of a product, 339, 349, 478-481, 486 
of the product of basic horizontal ma- 
trices, 343, 349 
of the product of basic vertical matrices, 
342-343, 349 
of the product of two basie matrices, 
341, 349 
of a product of two matrices, 478-481, 
486 
of sums of matrices, 348, 350 
Minor product, diagonal of, 176 
of e; vectors, 135, 138 
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inor product (cont.) 
а matrices with the same orthonormals, 


375-316, 381 i 
of a matrix and its general inverse, 
404, 430 Е 
süihimum rank of, 341-342, 349, 478- 
480, 486 


of orthonormals, 321, 329 
trace of, 178, 209 
of two basie matrices, minimum rank of, 
341-342, 349, 478-480, 486 
of two vectors, 115-116, 136-137 
factors of, 116, 136 
notation for, 116 
transpose of, 117-118, 137 
of type I supervectors, 126-129, 138 
transpose of, 127-129, 137 
of type II supervectors, as the product 
of two matrices, 143-144, 163 
of type III vectors, 229-232, 261 
notation for, 229-231, 261 
transpose of, 231-232, 261 
of type IV vectors, 240-248, 262-263 
diagrammatie representation, 241- 
244 
notation for, 240-241, 262 
numerical example of, 244-246 
transpose of, 246-248, 263 
Minor product moment, 122, 132, 137, 
166-168, 208 
of a basic matrix, rank of, 345, 350 
basic structure of, 373, 381 
of a data matrix, 288-291, 302-303 
computational checks, 289-291, 303 
computational steps, 288-289, 302 
of an e; vector, 135-136, 138 
left orthonormal of, 373 
powers of, 385, 399 
reduced, calculation of, 298-299 
trace of, 179, 209 
of a type III vector, 232-235, 261-262 
column, 234-235, 261-262 
row, 232-234, 261 
of a type IV vector, 248-252, 203 
column, 251-252, 263 
row, 249-251, 263 
of a unit vector, 132-133, 138 
Multiple products, order of multiplication 
for, 153-154, 164 
Multiplication, column-by-column, 434— 
436 
of diagonal matrices, 188-189, 211 
matrix, see Matrix multiplication 
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of a matrix, by a diagonal matrix, 184- 
190 
by an ei; matrix,.-205-207, 213 
by an e; vector, 183-184, 205-207, 
210, 213 
by an identity matrix, 197-198, 212 
by the inverse of a square matrix, 
436-445 
by the inverse of a symmetric matrix, 
453-460 
by the inverse of a triangular matrix, 
448-450 
by an orthonormal matrix, 322-325, 
329-330 
by a permutation matrix, 201-204, 
212-213 
by a sign matrix, 198-201, 212 
by its transpose, 166-169, 208 
by a unit vector, 182-183, 210 
order of, 153-154, 164 
row-by-row, 434-436 
scalar matrices in, 192-193, 211 
scalar quantities in, 193-194, 212 
of supermatrices, 217-267 
of a unit vector by a diagonal matrix, 
187-188 
of a unit vector by a permutation ma- 
trix, 204, 213 
of vectors, see Vector multiplication 


see also Product of a matrix and Product 
of matrices 


Multipliers, arbitrary, 476 
Multivariate analysis, 500 


Natural order, 52- 53, 59 
of a data matrix, 52, 59 
of a partial triangular matrix, 52 
of a rectangular matrix, 52, 59 
of a square triangular matrix, 53, 59 
and the supermatrix, 65- 66, 78 
and the transpose, 54, 60 
Nonbasic matrix, finding the rank of, 
356-357, 363 
general inverse of, 424-426, 432 
solution for, 493-. ——496, 500 
Noncomformable matrices, 147 
Nonhorizontal matrix, 365, 381 


Nonhorizontal АЕ | matrix, 365, 
381 


Nonsingular matrix, 405, 489 


Nonsquare basic matrix, general inverse 
of, 421-422, 432 
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Nonsymmetric partitioning, of symmetric 
simple matrices, 69 
Nonvertical orthonormal matrix, 381 
Normal vectors, 309-310, 328 
Null matrix, 42, 48 
transpose of, 57 
Null submatrix, 69 
Null vector, 46, 48 
Numerical subscripts, 22-23, 27-28 
to stand for separate matrices, 63 


Order, common, 147 
distinct, 148, 164 
of factors, 146, 163, 164 
common order, 147 
of the major product of two vectors, 134 
of a matrix, 20, 27 
of multiplication, 153-154, 164 
natural, see Natural order 
of a product moment matrix, 168, 208 
of products, 147-148, 164 
restrictions on, 327-328, 330 
specification of, 26-27 
of a supermatrix, 63, 78 
of a vector, 43, 48 
Orthogonal matrices, 309-331, 337, 349 
definition, 311, 328 
general, general inverse of, 408-409, 431 
orthogonal to one another, 326, 330 
orthonormal matrices from, 312, 329 
square, 314-317, 329 
product moment of, 314 
Orthogonal vectors, 309, 328 
Orthogonality theorem, 477, 486 
Orthonormal matrices, basic, 368 
inverse of, 407-408, 431 
left, see Left orthonormal 
major product moment of, 312-314, 
329-330 
matrices multiplied by, 3 
330 
minor product of, 321, 329-330 
nonhorizontal, 365, 381 
nonvertical, 381 
from orthogonal matrices, 312, 329 
powers of a major product moment of, 
320-321, 329 
product moment of, 316 
product of matrices with the same, 375- 
377, 381 
products of, 318-322, 329 
any number of square, 319-320, 329 


two square, 318-319 


22-325, 329- 


right, 365, 373, 384-388, 399 
square, see Square orthonormal matrices 


Parentheses, 17-18, 27 
Partial triangular matrix, 39-40, 48 
as a supermatrix, 71-73, 79 
Partitioned matrices, 64, 78 
Partitioning, 64, 78 
of an identity matrix, 69 
nonsymmetric, 69 
symmetric, 66-69, 78 
Partitioning vectors, 217-218, 260 
Pearson product moment correlation co- 
efficient, 275-276, 283 
Permutation matrix, 41, 48, 317, 329 
multiplication by, 201-204, 212-213 
product moment of, 203-204 
product of a general and, 357-359, 363 
unit vector multiplied by, 204, 213 
Postfactor, 116, 120, 476 
Postmultiplication, 116 
by a basic square matrix, 344-345, 349 
by a diagonal matrix, 186, 210, 301-302 
computational checks, 302, 304 
computations, 301-302, 304 
by an es; matrix, 205, 213 
by an e; vector, 184, 210 
by a permutation matrix, 202, 212 
of rectangles by square orthonormals, 
321-322, 329 
by a sign matrix, 198-199, 212 
by a unit vector, 182, 210 
by a vector, 180-181, 209 
Postmultiplier, 476 
Postpartitioning, 219-220, 260 
Postscript, 217, 260 
Powers, of a diagonal matrix, 190-192, 211 
of an identity matrix, 197-198, 212 
of a major product of orthonormals, 
320-321, 329 
of a matrix, 207-208, 213 
of the minor product moment, 385, 399 
of a product moment matrix, 377-378, 
381 
and basic structure, 374, 381 
trace of, 379-380, 382 
of a sign matrix, 199-201, 212 
Prediction, 9, 15, 487, 499 
Prediction formulas, 10-12, 15 
Prediction weights, 487, 499 
Predictor attributes, 10, 15 
Predictor measures, 499 
Predictor variables, 487, 499 
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Prefactor, 116, 120, = 
iplication, 11 
з cms matrix, 184-186, 210, 
300-301 
computational checks, 300 
computations for, 300 
by an ез matrix, 205-206, 213 
by an e; vector, 183, 210 


by а permutation matrix, 202-203, 212— 


213 
of rectangles by square orthonormals, 
322, 329 
by a sign matrix, 198, 212 
by a unit vector, 182, 210 
by a vector, 179-180, 209 
Pre- and postmultiplication, by a diagonal 
matrix, 187, 210 
of an e;; matrix, 206, 213 
by an e; vector, 184, 210 
by a permutation matrix, 203, 213 
by a sign matrix, 199, 212 
by a unit vector, 183, 210 
by a vector, 181-182, 209 
Prepartitioner, 218 
Prepartitioning, 218-219, 260 
Pre- and postpartitioning, 220-221, 260 
Prepartitioning vector, 218 
Prescript, 217, 260 
Principal diagonal, 21, 27 
Product, cumulative, 435 
factors of, 333 
major, see Major product 
of matrices, see Product of matrices 
of a matrix, see Product of a matrix 
maximum rank of, 338-339, 349 
minimum rank of, 339, 349 
minor, see Minor product 
order of, 147-148, 164 
of supermatrices, 259 
Product moment, correlation cor 
275-276, 283 
Product moment matrices, inequality of 
major and minor, 169-170, 208 
with the same orthonormals, 376-377, 
381 
Product moment matrix, 166, 208 
basic structure of, 373-375, 381 
diagonal of, 174 
factors of, 323-324, 329-330 
fractional powers of, 377-378, 381 
major, see Major product moment 
order of, 168, 208 


efficient, 
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of an orthonormal matrix, 316 
of a permutation matrix, 203-204 
powers of, see Powers, of a product 
moment matrix 
rank of, 345-346, 349-350 
of a square orthogonal matrix, 314 
trace of, and the basic diagonal, 379, 382 
involving square orthonormals, 324- 
325, 330 
Product of a matrix, by a diagonal matrix, 
184-188, 210 
by its general inverse, 496-499, 500 
by the inverse, of a basic square matrix, 
418-421, 431 
of a square matrix, 436-445 
of a symmetric matrix, computations 
for, 453-460 
ofa triangular matrix, computations 
for, 448-450 
by an orthonormal matrix, 322-325, 
329-330 
by a permutation matrix, 357-359, 363 
by a square orthonormal matrix, 380, 
382 
see also Multiplication 


Product of matrices, any two, rank of, 


480-481, 486 

basic horizontal, minimum rank of, 
343, 349 

basic vertical, minimum rank of, 342- 
343, 340 


diagonal, 188-189, 211 

i; matrices, 207, 213 

involving the identity matrix, 197, 212 

as major product of type II supervec- 
tors, 144-146, 163 

maximum rank of, see Maximum rank 

minimum rank of, see Minimum rank 

as minor product of type II supervec- 
tors, 143-144, 163 

order of, 147-148, 164 

orthonormal, 318-322, 329 

rank of, see Rank, of a product 

with the same orthonormals, 375-377, 
381 

with scalar Matrices, 192-193, 211 

with scalar quantities, 193-194, 212 

Square, see Square products of matrices 

Supermatrices, any number of, 259 

transpose of, 160-162, 165 

see also Multiplication 


Product of a vector and its transpose, 


121-123, 137 
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Product of vectors, scalar, 115-116 
undefined, 123-124, 137 
Products, basic square, basic structure of, 
367-368 
multiple, order of multiplieation for, 
153-154, 164 


Rank, of the major product moment, 346, 
350 
of a matrix, 334-330, 349 
computational procedure for finding, 
359-360, 363 
finding, 353-363 
with 0 rows or columns, 335-336, 349 
maximum, see Maximum rank 
minimum, see Minimum rank 
of a product, of any two matrices, 480— 
481, 486 
of matrices, 338-346, 349 
maximum, 338-340, 349 
minimum, 339, 341-343, 349 
involving square basic matrices, 343- 
345, 349 
of a product moment matrix, 345-346, 
349-350 
of products, special kinds, 339-343, 349 
of a sum of matrices, 346-348, 350 
of a supermatrix, 337-338, 349 
Rank theorem, 481-484, 486 
Raw covariance matrix, 277, 283 
Raw scores, correlation matrix from, 277- 
281, 283-284 
Rectangular matrices, orthogonal to one 
another, 326-327, 330 
Rectangular matrix, 30, 46 
transpose of, 56, 60 
Reduced minor product moment matrix, 
298-299 
Reduction equation, 476-477, 486 
Reduction procedure, 353-354, 357, 363 
Reduction vectors, 360, 361 
Regular inverse, 405, 430, 476 
Residual matrix, 360, 362, 363 
first, 391-392, 400 
Right orthonormal, 365, 373 
of a data matrix, 384—388, 399 
solution for, 387, 399 
Row and column sums, 286-287, 302 
Row subvector, 74 
Row sums, and square orthonormals, 325, 
330 
Row supervector, supermatrix as, 224- 
225, 261 
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type I, 74, 79 
type II, 75-76, 79, 89-90, 93 
type III, 224-225 
Row vector, 42, 75 
type III, see Type III row vector 
type IV, see Type IV row vector 
Row-by-row multiplication, 434—436 
Rows, as entities, 5-6 
Rows and columns, deletion of, 472-474, 
475 
interchangeability of, 6-8, 14-15 
meaning of, 14 


Scalar algebra, 36 
Scalar covariance, 283 
Scalar factors and commutativity, 194, 212 
Scalar matrices, 35-36, 47 
in matrix multiplication, 192-193, 211 
sums and differences of, 107, 112 
transpose of, 57, 60 
Scalar product of two vectors, 115-116 
Scalar quantities, 46, 49 
in matrix multiplication, 193-194, 212 
transpose of, 58 
Scaling constant for iterated vectors, 389- 
390 
Second-order matrix, inverse of, 429-430, 
432 
Second-order supermatrix, inverse of, 465- 
472, 474-475 
Sign matrix, 37, 47, 317, 329 
inverse of, 407, 431 
multiplication by, 198-201, 212 
postmultiplication by, 198-199, 212 
powers of, 199-201, 212 
pre- and postmultiplication by, 199, 
212 
premultiplication by, 198, 212 
transpose of, 57, 60 
Sign vector, 45, 48 
Signs, of elements in the basic diagonal, 
368-369, 381 
Simple matrices, 62 
definition of, 78 
subtraction of, 104-105, 112 
transpose of, 89-90, 93 
Singular matrix, 405 
Size of a matrix, 19-21, 27 
Skew symmetric matrix, 38, 47, 109, 112 
Solution for basic structure, for the basic 
diagonal, 388 
for the left orthonormal, 392-393, 400 
for the right orthonormal, 387, 399 
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ion for linear equations, general in- 
jog wee 488, 499, 500 
for a horizontal basic matrix, 492-493, 
500 
if rank of z is known, 494-495, 500 
when z is not basic, 493-496, 500 
when z is square and basic, 489, 499 
when т is vertical and basic, 490-492, 
499-500 
Solutions for linear equations, infinite 
number of, 500 
Specification of order, 26-27 
Square basic matrices, inverse of, 412-416, 
431 
inverse of the product of, 405-406, 430 
rank of products involving, 343-345, 349 
Square matrix, 20, 30-31, 46 
and its transpose, sum of, 108-109, 112 
Square orthogonal matrix, 314-317, 329 
product moment of, 314 
Square orthonormal matrices, product of 
any number, 319-320, 329 
product of two, 318-319 
Square orthonormal matrix, column sums 
and, 325, 330 
construction of, 314-315 
inverse of, 407, 431 
postmultiplication of rectangular by, 
321-322, 329 
premultiplication of rectangular by, 322, 
329 
product moment of, 316 
produet of а matrix by, 380, 382 
row sums and, 325, 330 
special types, 317-318, 329 
trace of product moment involving, 
324-325, 330 
from a vector, 317-318, 329 
Square products of matrices, 171-173, 209 
basic, basic structure of, 367-368 
factors of, 171-172, 209 
major and minor, 172-173, 209 
Square root, of a diagonal matrix, 191, 211 
Standard deviations, 34 
Standard measures, basic structure for, 
397-398, 400 
Submatrix, 63 
diagonal, 67 
null, 69 
Subscript, external, 75 
to indicate order, 55, 60 
letter, 23-25, 28 
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numerical, 22-23, 27-28 
to stand for separate matrices, 63 
in a symmetric matrix, 32 
vector, 220 
Subtraction, of simple matrices, 104-105, 
112 
of supermatrices, 105-112 
Subvectors, 73-74 
Sum of a square matrix and its transpose, 
108-109, 112 
Sum of matrices, rank of, 346-348, 350 
Sum of two matrices, inverse of, 428-429, 
432 
Sums and differences of matrices, diagonal, 
107, 112 
scalar, 107, 112 
symmetric, 107-108, 112 
transposes of, 106-107, 112 
Superdiagonal matrix, 69-70 
Supermatrices, addition of, 105-106, 112 
equality of, 76-77, 79 
multiplication of, 217-267 
product of any number of, 259 
subtraction of, 105, 112 
Supermatrix, 62-81 
as a column supervector, 223-224, 260- 
261 
definition of, 62, 78 
expanded notation for, 221-222 
general diagonal, 69-70, 79 
height of, 63 
inverse of, 464-475 
natural order of, 65-66, 78 
order of, 63, 78 
partial triangular matrix as, 71-73, 79 
rank of, 337-338, 349 
as à row supervector, 224-225, 261 
second-order, inverse of, 465-472, 474- 
475 
superorder of, 64 
transpose of, 82-95 
notation for, 225-229, 261 
width of, 63 
Superorder, 106 
of a supermatrix, 64 
Supervector, column, see Column supe! 
vectors 
Tow, see Row supervectors 
type I, see Type I supervectors 
type II, see Type II supervectors 
type III, see Type III vectors 
type IV, see Type IV vectors 
Supradiagonal elements, 33, 85 
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Symmetric matrices, sums and differences 
of, 107-108, 112 
Symmetric matrix, 31-33, 47 
basic, inverse of, 416—418 
inverse of, computations for, 460—463 
subscripts in, 32 
transpose of, 57-58, 60 
Symmetric nonbasic matrix, general in- 
verse of, 407 
Symmetric partitioning, 66-69, 78 
of a diagonal simple matrix, 68-69 
of a symmetric simple matrix, 67-68, 78 
Symmetric simple matrix, nonsymmetric 
partitioning of, 69 
symmetrically partitioned, 67-68, 78, 258 
transpose of, 84-86, 92 


Theorem, general matrix reduction, 476— 
486 
special cases of, 484-486 
minimum rank of a product of two ma- 
trices, 478-481, 486 
orthogonality, 477, 486 
rank, 481-484, 486 
Thurstone, L. L., Multiple Factor Analysis, 
324 
Trace, 173-179, 209 
computational notation for, 177-179 
of a diagonal matrix, 379-380, 382 
of a major product, 178, 209 
of a major product moment, 179, 209 
of a minor product, 178, 209 
of a minor product moment, 179, 209 
of powers of product moments, 379-380, 
382 
of a product moment matrix, 379, 382 
and the basic diagonal, 379-380, 382 
and square orthonormals, 324-325, 
330 
Transpose, of a diagonal supermatrix, 86— 
87, 92 
difference between a matrix and its, 
109-111, 112 
of an identity matrix, 57, 60 
of a major product, of two type I super- 
vectors, 131-132, 137 
of two vectors, 120-121, 137 
of type III vectors, 236-237, 262 
of type IV vectors, 255-256, 263 
of a matrix, 51-81 
definition, 53-54, 59-60 
symbol for, 54-55, 60 
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use of subscripts to indicate order, 
55, 60 
of the minor product of two vectors, 
117-118, 137 
of a minor product of type I super- 
vectors, 127-129, 137 
of a minor product of type III vectors, 
231-232, 261 
of a minor product of type IV vectors, 
246-248, 263 
multiplication of a matrix by, 166-169, 
208 
of a product of matrices, 160-162, 165 
the product of a vector and its, 121-123, 
137 
of a rectangular matrix, 56, 60 
of a scalar matrix, 57, 60 
of a scalar quantity, 58 
of a sign matrix, 57, 60 
of a simple matrix, expressed as a type 
II supervector, 89-90, 93 
sum of a square matrix and, 108-109, 
112 
of sums and differences of matrices, 106— 
107, 112 
of a supermatrix, 82-95 
notation for, 225-229, 261 
rules for, 83-84, 92 
of а symmetric matrix, 57-58, 60 
of a symmetrical partitioned sym- 
metric simple matrix, 84-86, 92 
of a transpose, 56, 60 
of a triangular matrix, 56, 60 
of a type I supervector, 87-88, 93 
of a type III column vector, 225-226, 
261 
of a type III row vector, 226-227, 261 
of a type III vector, 91-92, 93 
of a type IV vector, 228-229, 261 
of а vector, 58-59 
Triangular factors, 353, 362 
construction of, 354-356, 363 
Triangular matrix, 38-39, 47-48, 337, 349 
inverse of, 409-412, 431 
computations for, 450-453 
lower, 39, 47 
transpose of, 56, 60 
upper, 38, 48 
Type I supervectors, 73-74, 79 
column, 73, 79 
major product of, 129-131, 137 
transpose of, 131-132, 137 
minor product of, 126-129, 138 
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Type I supervectors (cont.) 
row, 74, 79 к 
transpose оѓ, 127-129, 137 
transpose of, 87-88, 93 T 
II supervectors, 74—76, 7€ 
мее 24-15, 79, 88, 93 
major product of, as a product of two 
matrices, 144—146, 163 
matrix expressed as, 142-143, 162-163 
minor product of, as a product of two 
matrices, 143-144, 163 
row, 75-76, 79, 89-90, 93 
ав transposes of simple matrices, 88- 
90, 93 
Type III column vectors, major product 
moment of, 237-239, 262 
minor product moment of, 234-235, 
261-262 
in a row type IV supervector, 224-225 
transpose of, 225-226, 261 
Type III row vectors, in a column type 
IV supervector, 223-224 
major product moment of, 239-240, 262 
minor product moment of, 232-234, 261 
transpose of, 226-227, 261 
Type III vectors, 78, 79 
expanded notation for, 221 
major product moment of, 237-240, 262 
major product of, see Major product, of 
type III vectors 
matrie elements in, 78 
minor product moment of, 232-235, 
261-262 ` 
minor product of, see Minor product, of 
type III vectors 
simplified notation for, 223 
transpose of, 91-92, 93 
Туре IV column vectors, major product 
moment of, 257-259, 263-264 
minor product moment of, 251-252, 263 
type III row vectors in, 223-224, 260 
Type IV row vectors, major product 
moment of, 256-257, 263 
minor product moment of, 249-251, 263 
type III column vectors in, 224-225, 261 
Type IV vectors, 223-225, 260-261 
major product moment of, 256-259, 
263-264 
major product of, see Major product, 
of type IV vectors 
minor product moment of, 248-252, 263 
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minor product of, see Minor product, 
of type IV vectors 
transpose of, 228-229, 261 


Undefined products of vectors, 123-124, 
137 
Unit vector, 44-45, 48 
major product moment of, 133-134, 138 
minor product moment of, 132-133, 138 
multiplication of a matrix by, 182-183, 


210 

multiplication by a diagonal matrix, 
187-188 

multiplication by a permutation matrix, 
204, 213 


postmultiplication by, 182, 210 
pre- and postmultiplication by, 183, 210 
premultiplication by, 182, 210 


Variables, 6 
criterion, 487, 499 
Predictor, 487, 499 
Variance, 269-271, 282 
of а set of measures from their mean, 
271, 283 
ав а special case of covariance, 275, 283 
Vector, arbitrary, 387, 399 
column, 42, 48, 75 
type III, see Type III column vector 
type IV, see Type IV column vector 
definition of, 42, 48 
ĉi, see e; vector and e; vectors 
elementary, 45 
kinds, 44-46, 48 
. major product of two, see Major product 
of two vectors 
with matrix elements, 78, 79 
of means, 277, 283 
when N is very large, 296, 303 » 
multiplied by its transpose, 121-123, 137 
normal, 309-310, 328 
null, 46, 48 
order of, 43, 48 
orthogonal, 309, 328 
Partitioning, 217-218, 260 
Postmultiplication by, 180-181, 209 
pre- and Postmultiplication by, 181- 
182, 209 
premultiplication by, 179-180, 209 
Prepartitioning, 218 
reductjon, 360, 361 
TOW, 42, 75 
type III, see Type TIT row vector 


INDEX 


Vector (cont.) 
type IV, see Type IV row vector 
sign, 45, 48 
square orthonormal matrix from, 317- 
318, 329 
transpose of, 58-59 
type III, see Type III vectors 
type IV, see Type IV vectors 
undefined products of, 123-124, 137 
unit, see Unit vector 
zero, 46, 48 
zero-one, 45, 48 
Vector multiplication, 115-141 
and the commutative law, 124-126, 137 
row-by-row or column-by-column, 434- 
436 
Vector subscripts, 220 
Vectors, iterated, scaling constant for, 
389-390 
minor product of two, see Minor product 
of two vectors 
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order of major product of two, 134 
Vertical basic matrix, 344, 349 
Vertical matrices, basic, minimum rank for 
product of, 342-343, 349 
Vertical matrix, 20, 27 
major product moment of, 328, 330 


Weights, 11 
prediction, 487, 499 
Width, of a matrix, 20, 27 
of a supermatrix, 63 
Worksheets, 26, 28 
for computing inverses, 433 
designing, 435 


Young, Gale, 367 


Zero matrix, 42, 48 

Zero vector, 46, 48 
Zero-one matrix, 40-41, 48 
Zero-one vector, 45, 48 


Form No. 3. 
PSY, RES.L-1 


Bureau of Educational & Psychological 
Research Library. 


The book is to be returned within 
the date stamped last. 


WBGP-59/60-5119C-5M 


